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I.  OVERVIEW 


A.  Summary 

EEG  Systems  Laboratory  develops  and  applies  advanced  technologies  for  measuring  and  imaging  neu- 
rocognitive  signals  of  the  human  brain.  During  the  three-year  funding  period,  4/1/87  to  3/31/90,  a 
number  of  objectives  were  accomplished.  First,  we  completed  an  experiment  which  measured,  for  the 
first  time,  the  effects  of  sustained  mental  work  on  functional  networks  of  the  human  brain.  In  addi¬ 
tion  to  the  well-known  global  decrease  in  cortical  activation  with  operational  fatigue,  we  found  signs  of 
differential  alteration  of  cortical  networks  during  split-second  pre-  and  post-stimulus  intervals  of  a  dif¬ 
ficult  visuomotor —working  memory  task.  The  pattern  from  the  prestimulus  interval,  which  required 
maintainence  of  previous  stimuli  in  memory,  changed  most  dramatically  during  hours  7-9,  which  was 
before  performance  had  significantly  declined.  The  pattern  associated  with  inhibiting  a  response,  when 
the  current  stimulus  matched  the  remembered  stimulus  from  two  trials  back,  changed  most  in  the 
final  two  hours  of  the  session.  The  pattern  associated  with  feature  registration  of  the  visual  stimuli 
was  least  affected.  It  seems  feasible  to  further  develop  this  technology  into  an  early-warning  system 
to  detect  immanent  transient  cognitive  lapses.  The  report  of  these  findings  has  been  accepted  for 
publication  by  the  journal  Electroencephalography  and  Clinical  Neurophysiology,  and  the  preprint  is 
included  in  this  report. 

Second,  we  conducted  an  experiment  to  measure  functional  cortical  patterns  during  elementary  gra- 
phemic,  phonemic,  semantic,  and  grammatical  tasks.  Nine  59-channel  EEG  recordings  were  made, 
and  an  initial  analysis  has  been  completed.  Spatial  and  temporal  evoked  potential  features  dis¬ 
tinguished  letter  from  non-letter,  word  from  non-word,  and  syntactic  from  non-syntactic  processing. 
Differences  between  linguistic  and  non-linguistic  conditions  were  mainly  left-sided.  The  three 
language  conditions  exhibited  sharply  localized  effects  over  left  temporal  and  frontal  cortex,  with 
differences  between  the  processing  of  closed  class  (functional,  syntactic)  versus  open  class  (content) 
words.  Evidence  for  bilaterality  of  semantic  (lexical)  function  was  also  found,  as  was  evidence  of 
priming  of  visual  association  cortex  for  processing  unfamiliar  graphic  figures. 

Third,  several  technological  advances  were  made  during  the  past  three  years.  We  expanded  our  EEG 
recording  and  analysis  capabilities  from  64  to  128  channels.  Data  inspection  and  analysis  has  been 
assisted  by  the  "Scientists’  Workbench",  an  interactive  exploratory  data  analysis  and  display  program. 
Other  technical  developments  include  development  of  methods  and  programs  for  analyzing  magnetic 
resonance  images  (MRIs)  in  order  to  construct  three-dimensional  models  of  each  subject’s  brain. 
These  models  are  used  to  improve  functional-anatomical  localization,  both  for  equivalent  current 
dipole  analysis  and  improved  spatial  deblurring  procedures.  Applying  dipole  analysis  to  octant  visual 
steady  state  stimuli  produced  sources  located  near  the  occipital  pole  contralateral  to  side  of  stimula¬ 
tion  and  inverted  about  the  horizontal.  These  results  are  comparable  to  intercerebral  locations 
reported  for  early  latency  transient  EPs,  as  well  as  for  equivalent  dipoles  computed  from  the  MEG. 
However,  the  EEG  equipment  for  this  type  of  dipole  analysis  costs  about  100  times  less  than  MEG 
equipment  with  a  comparable  number  of  channels.  Although  this  method  proved  very  efficient  and 
effective  in  locating  dipoles  for  simple  steady-state  sensory  stimuli,  single  equivalent  dipoles  are  not  a 
suitable  model  for  the  complex,  distributed  processes  of  cognitive  functions.  Work  is  underway  at 
EEGSL  on  the  development  of  more  realistic  models.  Our  3-D  brain  imaging  and  deblurring  tech¬ 
niques  are  described  in  a  preprint  included  in  this  report,  titled  'Beyond  Topographic  Mapping: 
Towards  Functional- Anatomic  Imaging  with  124-Channel  EEG’s  and  3-D  MRI’s". 
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Summary  Topographic  patterns  of  event-related  covariance  between  electrodes  were  measured  from  subjects  performing  a 
difficult  memory  and  fine-motor  control  task  for  10-14  h.  Sinking  changes  occurred  in  ihe  patterns  after  subjects  performed  the  task 
for  an  average  of  7-9  h.  but  before  performance  deteriorated.  Pattern  strength  was  reduced  in  a  fraction-of-a-second-long  response 
preparation  interval  over  midline  precentral  areas  and  over  the  entire  left  hemisphere.  By  contrast,  pattern  strength  in  a  succeeding 
response  inhibiuon  interval  was  reduced  over  all  areas.  The  pattern  changed  least  in  an  intervening  interval  associated  with 
visual-sumulus  processing.  This  suggests  that,  in  addition  to  the  well-known  global  reduction,  in  neuroeiectnc -aignal  strength, 
functional  neural  networks  are  selectively  affected  by  sustained  mental  work  in  specific  fraction-of-a-second  task  intervals. 

Key  words:  Prolonged  mental  work:  Functional  neural  networks;  Event-related  covariances:  Event-related  potentials:  Laplacian 
derivation;  Spatio-temporal  maps:  Contingent  negative  variauon:  N1-P2;  P30O 


The  high  incidence  of  serious  accidents  caused 
by  fatigued  workers  provides  compelling  motiva¬ 
tion  for  study  of  the  deleterious  effects  of  pro¬ 
longed  mental  work.  Although  psychological  theo¬ 
ries  have  been  advanced  to  explain  how  prolonged 
mental  work  affects  sensory,  motor  and  higher 
cognitive  functions  (Warm  and  Parasuraman 
1987),  spatially  and  temporally  detailed  measure¬ 
ments  of  the  associated  changes  in  brain  activity 
needed  to  test  these  theories  have  been  lacking. 

Some  electroencephalographic  correlates  of  de¬ 
creasing  alertness  have  been  described,  for  exam¬ 
ple,  replacement  of  alpha  band  activity  by  varia¬ 
ble  fast  frequency,  low  voltage  rhythms  (Roth 
1962;  Williams  et  ai„  1962;  Kellaway  and 
Maulsby,  1967;  Walter  et  ai„  1967),  increase  in 
delta  and  theta  band  power  (Lubin  et  aL,  1969), 
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and  an  increase  in  the  theta- to-alpha  band  power 
ratio  (Matousek  1967;  Gevins  et  al..  1977;  re¬ 
viewed  in  Matejcek  1982).  Hori  (1985)  reported 
increases  in  alpha  wave  amplitude  over  central 
and  posterior  sites  and  decreases  over  frontal  sites 
in  the  transition  from  waking  to  sleep.  A  study  by 
Naitoh  et  al.  (1971)  demonstrated  that  the  contin¬ 
gent  negative  variation  (CNV)  disappears  with 
sleep  deprivation.  AguirTe  and  Broughton  (1987) 
reported  that  the  P300  wave,  but  not  the  CNV.  is 
reduced  during  daytime  sleepiness.  Mane  et  al. 
(1983)  also  showed  that  the  P300  peak  is  particu¬ 
larly  sensitive  to  prolonged  performance  of  a  task. 
None  of  these  studies,  however,  attempted  to  ob¬ 
tain  temporally  and  spatially  detailed  neuroelec¬ 
tric  measurements  from  subjects  performing  dif¬ 
ficult  cognitive  tasks  over  extended  periods  of 
time. 

The  goal  of  the  present  study  was  to  measure 
the  effect  of  sustained  mental  work  on  the  func¬ 
tional  topography  of  the  human  brain  (Gevins  and 
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Bressler  1988).  We  used  event-related  covariance  about  6  h,  until  the  learning  curves  for  response 

(ERC)  analysis  (Gevins  et  al„  1987,  1989a,  b)  to  error  stabilized.  Subjects  began  at  about  1.30  p.m. 

measure  neuroelectric  patterns  related  to  perfor-  the  following  day,  and  during  the  ensuing  10-14  h 

mance  of  a  difficult  memory  and  fine-motor  con-  performed  several  types  of  tracking  and  cognitive 

trol  task  during  a  10-14  h  period.  The  task  re-  tasks,  including  about  1400  visuomotor  mimory 

quired  that  subjects  monitor  a  low  event  rate  (11  task  trials.  The  session  consisted  of  a  5~8^work  S'%  K  A.  'C 

events/min),  single  modality  (visual)  stimulus  pre-  period,  a  brief  dinner  break,  then  another  5-7  h 

sented  from  a  single  source  (video  screen),  and  work  period  which  ended  when  the  subject  was 

make  successive  cognitive  discrimination  judg-  too  exhausted  to  continue. 

ments  (Davies  and  Parasuraman  1982).  It  involved  Visuomotor  memory  task  trials  were  presented 

many  processes  known  to  be  affected  by  pro-  in  blocks  of  50.  Except  for  the  first  2  trials  in  the 

longed  mental  work,  such  as  expectation  and  pre-  block,  subjects  produced  a  precise  finger  pressure 

paration,  stimulus  processing,  short-term  memory  proportional  to  a  number  seen  2  trials  back  (about 

and  attention  (Matejcek  1982).  The  effects  of  11.5  sec  earlier).  Each  trial  began  with  the  disap- 

practice  and  variations  in  performance  accuracy,  pearance  of  the  letter  X  from  the  video  screen.  A 

which  can  degrade  the  quality  of  sustained  atten-  single-digit  stimulus  number,  which  the  subject 

tion  in  successive  discrimination  tasks  (Parasura-  was  required  to  remember,  appeared  750  msec 

man  et  al.  1987),  were  carefully  controlled  so  as  later.  Stimuli  were  presented  for  325  msec  on  a 

not  to  confound  the  interpretation  of  neurophysio-  Videographics-11  amber  CRT  monitor  70  cm  from 
logical  data.  the  subject.  Stimuli  subtended  a  visual  angle  un¬ 

der  1.5°,  with  an  illumination  of  11  cd/nr  against 
a  background  of  (3^1  cd/m:.  When  the  stimulus  0.|  Cj/p~ 
Methods  number  disappeared,  the  subject  was  required  to  1 

produce  a  pressure  of  0.1 -0.9  kg  with  his  right 
Subjects  and  task  index  finger  on  an  isometric  pressure  transducer. 

Each  of  5  healthy,  right  handed,  male  Air  Force  The  appropriate  response  pressure  corresponded 

jet  test  pilots  practiced  a  battery  of  4  tasks  for  to  the  stimulus  number  presented  2  trials  earlier. 

VISUOMOTOR  MEMORY  TASK  (VMMT) 

Fist  Second  TNrd  Fourth  Fifth  Sixth 

Tna!  Tnai  Trial  Trial  Trial  Trial  _ 


3  7  ft  9  5  1 

•  *  1 _ 

I  1  i 

1  1  I 

1 _ |  i 

*  i  ♦ 

Non*  Nor*  _.r  7.1  None  8.9 


StKTxJuS 

Pressed 


Sue  set's 
Resooose 


Presented  OK  OK  3.5  7.1  OK  8.6 

Fie.  I.  The  visuomotor  memory  task  (VMMT)  requirrd  Out  sunects  remember,  during  each  tnai.  a  stimulus  number  from  2  tnals 
back,  an  J  produce  ,  precise  nght  index  linger  pressure  in  respon  d  to  that  number.  Each  tnai  consisted  of  a  warning  (disappearance 
of  the  ter  X  f.-cm  the  video  screen  -  ot  shown),  followed  0T5  sec  later  by  presentation  of  a  single-digit  visual  sumulus  to  be 
remenb*  ed.  fol  owed  by  the  subject's  finger- press;  re  response  to  the  sumulus  number  presented  2  tnals  back,  followed  1  sec  after 
me  p-  ak  of  the  rtsponse  bv  presentauon  cf  a  2-<ligii  feedback  .lumber  indicating  the  accuracy  of  the  response.  The  feedback  number 
was  ui.der  ned  when  the  subject's  response  was  rugt.ly  accurate.  Subjects  were  required  to  withhold  their  response  when  the  current 
sutrul;  -s  number  was  identical  to  the  stimulus  2  tnals  back.  (No  response  was  required  on  the  first  2  tnals  of  each  block  of  50  tnals  - 

signified  by  'OK'). 
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For  example,  if  the  stimulus  numbers  in  4  succes¬ 
sive  tn-is  were  3,  7,  6.  and  9,  the  correct  responses 
would  be  0.3  kg  to  the  6  and  0.7  kg  to  the  9  (Fig. 
1).  A  two-digit  feedback  number  "  as  presented  in 
each  trial  1  sec  after  the  peak  of  the  response  to 
indicate  the  actual  pressure  exerted.  The  feedback 
number  was  underlined  if  the  response  was  close 
to  the  required  response,  based  on  an  'adaptive 
error  tolerance',  computed  as  the  geometric  mean 
of  the  response  error  (distance  from  the  required 
pressure  response)  on  the  previous  5  trials. 

To  increase  the  difficulty,  the  subject  was  re¬ 
quired  to  withhold  his  response  in  the  approxi¬ 
mately  20^  of  the  trials  when  the  stimulus  number 
was  the  same  as  the  number  from  2  trials  back. 
For  example,  if  the  numbers  6,  9.  6  were  pre¬ 
sented,  the  subject  was  not  to  respond  to  the 
second  6  (Fig.  1). 

Recording 

Neuroelectric  data  were  recorded  from  an  array 
of  electrodes  built  into  a  nylon  mesh  hat.  Signals 
were  recorded  from  27  electrodes  and  were  refer¬ 
enced  to  the  right  mastoid.  All  signals  were 
amplified  by  a  Bioelectric  Systems  Model  AS-64P 
with  0.016-30  Hz  passband  and  digitized  to  11 
bits  at  128  Hz.  Vertical  and  horizontal  eye  move¬ 
ments  (VEO,  HEO),  activity  of  the  right  flexor 
digitorum  muscle  (EMG),  cardiac  rate  (EKG), 
respiration,  and  EEG  activity  at  left  mastoid  were 
recorded  from  all  subjects.  Two  technicians,  work¬ 
ing  independently,  edited  the  time-senes  traces 
off-line  and  eliminated  data  wrih  evidence  of  con¬ 
tamination  by  potentials  generated  by  eye  move¬ 
ments,  muscle  activity,  or  instrumental  noise 
sources.  An  optimal,  least-squares  estimate  of  the 
laplacian  operator  was  computed  to  reduce  spatial 
low-pass  distortion  resulting  from  transmission 
through  the  skull  and  scalp.  Electrodes  at  the  edge 
of  the  array  were  eliminated  from  the  analysis 
because  it  is  not  possible  to  reliably  estimate  their 
laplacian  (Gevins  et  al.  1989a).  Even  though  lapla- 
cian-transformed  potentials  are  proportional  to 
current  density,  we  will  refer  to  the  wave  forms  as 
potentials,  in  keeping  with  common  usage. 

Recording  sessions  were  divided  into  3  epochs 
termed  Early.  Middle,  and  Late  based  on  response 
error  (Fig.  2).  Each  epoch  was  long  enough  to 


allow  us  to  form  stable  estimates  of  the  functional 
brain  patterns  occurring  dunng  it.  The  Early  epoch 
lasted  approximately  from  1.30  p.m.  to  8.30  p.m.. 
the  Middle  from  8.30  p.m.  to  10.30  p.m..  and  the 
Late  from  11.30  p.m.  to  1.30  a.m. 

Signal  analysis 

ERCs,  a  statistical  measure  of  functional  inter¬ 
relatedness  between  the  time  senes  of  2  channels, 
were  computed  from  pre-  and  poststimulus  inter¬ 
vals  of  the  task.  Although  the  exact  brain  sources 
of  scalp  ERCs  have  not  yet  been  determined, 
intracerebral  recording  from  animals  (John  et  al. 
1973;  Freeman  1975;  Bressler  1987a.  b)  and  scalp 
recordings  from  humans  (Gevins  1987a;  Gevins 
and  Bressler  1988;  Gevins  et  al.  1‘  9a)  suggest 
that  ERC  patterns  reflect  the  coordinated,  low- 
frequency,  task-related  dendntic  activity  pattern 
of  distributed  cortical  neuronal  networks. 

ERCs  were  computed  from  bandpass-filtered 
event-related  potentials  (ERPs)  time-locked  to  the 
presentation  of  the  numeric  stimulus,  and  aver¬ 
aged  over  the  5  subjects.  Three  well-known  ERP 
waves  were  used  as  timing  markers:  the  prestimu¬ 
lus  CNV  dunng  the  Finger- Pressure  tnals,  the 
poststimulus  N1-P2  peaks  during  stimulus  proces¬ 
sing  on  the  Finger- Pressure  trials,  and  the  post¬ 
stimulus  P3  elicited  dunng  the  infrequent  No-Press 
trials.  The  prestimulus  ERC  interval  was  500  msec 
wide  and  was  centered  312  msec  before  the 
numeric  stimulus  (Fig.  3A).  The  2  poststimulus 
ERC  intervals  were  187  msec  wide.  The  first  was 
centered  at  125  msec  after  the  stimulus  for 
Finger- Pressure  trials  (Fig.  4A)  and  the  second  at 
375  msec  for  No-Press  tnals  (Fig.  5A). 

Bandpass  filters  were  chosen  :o  enhance  the 
ERP  waves  of  interest.  Thus  the  prestimulus  inter¬ 
val  was  filtered  with  a  0.01-3  Hz  passband  to 
emphasize  the  CNV.  whereas  a  4-7  Hz  passband 
was  selected  for  the  poststimulus  intervals  to 
accentuate  the  N1-P2  and  P3  components.  The 
cross-covariance  function  was  computed  to  16  lags 
(+/-  125  msec)  for  the  0.01-3  Hz-filtered  inter¬ 
val.  and  to  8  lags  (-/-62  msec)  for  the  4-7 
Hz- filtered  intervals.  The  ERC  was  defined  as  the 
maximum  absolute  value  of  the  lagged  time  senes 
cross-covariance  function.  EP.O  for  the  3  inter¬ 
vals  were  computed  between  all  155  pairwise  com- 
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binations  of  the  18  non-edge  scalp  electrodes  com¬ 
mon  to  all  5  subjects. 

Statistical  evaluation 

To  measure  functional  changes  in  the  brain 
strictly  related  to  sustained  mental  work  and  un¬ 
confounded  by  variations  in  performance  accu¬ 
racy,  we  controlled  for  the  significant  effect  of 
response  error  across  the  Early,  Middle  and  Late 
epochs.  The  most  stringent  method  was  to  analyze 
only  the  most  accurate  trials  from  each  of  the  3 
trial  sets. 

We  first  selected  the  most  accurate  half  of  each 
subject's  Early  trials,  and  then  selected  trials  with 
comparable  accuracy  from  corresponding  Middle 
and  Late  sets.  Accurate  trials  were  those  with 
response  error  less  than  the  mean  response  error 
of  the  Early  trial  set.  We  then  discarded  the  least 
accurate  outlier  trials  of  the  accurate  Middle  and 
Late  subsets  (roughly  2%  of  the  trials)  so  that  the 
final  accuracy  distributions  of  the  three  epochs 
did  not  significantly  differ  ( P  >  0.2  by  the  Stu¬ 
dent's  t  test)  The  numbers  of  total  trials  in  the 
resulting  accurate  trial  sets  were  599  (Early),  520 
(Middle i,  and  402  (Late)  (Fig.  2).  Measures  of 
motor  performance  did  not  differ  between  these 
final  accurate  trial  sets.  We  also  selected,  from  all 
fr<i^z  3  epochs,  No-press  trials  in  which  the  subjects 
correctly  inhibited  their  response. 

We  determined  the  significance  of  Early,  Mid¬ 
dle.  and  Late  ERCs  by  referring  to  the  standard 
deviation  of  the  ERC  ‘noise’  distributions  ob¬ 
tained  with  a  randomization  procedure  from  the 
same  trial  sets.  Significant  ERCs  were  those  with  a 
probability  of  being  less  than  0.05  from  the  ‘noise' 
distribution.  ERC  significances  in  each  interval 
were  displayed  on  computer  graphic  images  of  a 
model  head’  the  thickness  of  a  displayed  ERC  line 
is  proportional  to  the  negative  log  of  significance. 
The  relative  involvement  of  each  electrode  site  in 
an  ERC  pattern  was  determined  by  taking  the 
sum  of  all  significant  ERCs  in  which  it  par¬ 
ticipated.  and  displaying  at  that  site  a  disk  whose 
size  was  proportional  to  the  sum. 

Analysis  of  variance  (ANOVA)  of  the  mean 
absolute  ERC  value  (called  ERC  magnitude  in 
Table  I)  provided  an  index  of  the  change  in  mean 
level  of  between-site  covariation  over  Early,  Mid¬ 


dle.  and  Late  epochs.  To  compare  any  2  epochs 
for  differences  in  pattern  configuration  (called 
ERC  pattern  in  Table  I),  the  2  patterns  of  signifi¬ 
cant  ERCs  were  correlated.  The  ERCs  in  each  set 
were  pooled  from  the  group  of  site  pairs  that  had 
significant  ERCs  in  either  pattern.  We  used  a 
distribution-independent  ‘Monte  Carlo’  procedure 
to  estimate  correlations  between  patterns  (Efron 
1982)  since  we  could  not  confirm  that  the  data 
were  normally  distributed  after  examining  the 
small  number  of  repeated  measures.  To  compare 
the  significance  of  ERC  magnitude  differences 
with  ERP  amplitude  differences,  we  tested  the 
change  in  root-mean-square  amplitude  over  epochs 
for  each  ERP  wave  by  ANOVA  (ERP  amplitude 
in  Table  I).  Pairwise  comparisons  of  ERC  magni¬ 
tude  and  ERP  amplitude  were  performea  by  the 
Scheffe  post-hoc  procedure. 

ERC  patterns  of  individual  subjects  were  com¬ 
pared  by  using  artificial  ‘neural  network’  pattern 
classification  procedures,  based  on  the  trial-by- 
trial  discnminabihtv  of  Early  and  Middle  pre¬ 
stimulus  intervals  (Gevins  1980.  1987b;  Gevins 
and  Morgan  1986.  1988;  Gevins  et  al..  1979,  1981. 
1983.  1985.  1987.  1989a.  b).  Single-tnal  ERCs. 
computed  for  channel  pairs  of  the  average  ERC 
patterns  were  considered  as  possible  variables.  The 
ratio  of  observations  (tnals)  to  variables  (ERCs) 
exceeded  10:1  in  each  analysis.  Significance  of 
the  average  test  set  classification  was  assessed  by 
reference  to  the  binomial  distribution. 


Results 

Behavioral  analysis 

By  ANOVA,  no  significant  differences  were 
found  between  three  100-tnal  subsets  at  the  begin¬ 
ning,  middle  and  end  of  the  Early  epoch.  This 
suggested  that  performance  had  asymptoted  and 
that  any  subsequent  performance  degradation 
could  be  attributed  to  prolonged  mental  work.  We 
observed  that  performance  changed  from  Early  to 
Middle  to  Late  epochs  (Fig.  2)  with  fewest  errors 
occurring,  as  expected,  in  trials  from  the  Early, 
epoch.  ANOVA  revealed  a  significant  increase  in 
response  error  from  Early  to  Late  epochs  ( F  (2. 
12)  =*  12.5.  P  <  0.005)  (Fig.  2);  Scheffe  post-hoc 
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Fig.  2.  Mean  and  standard  deviation  of  response  error  for 
Early.  Middle,  and  Late  epochs.  Upper  set  of  points  represents 
ail  Finger- Pressure  trials,  while  lower  set  is  from  the  most 
accurate  Finger-Pressure  trials. 


comparison  revealed  that  the  differences  were  sig¬ 
nificant  between  Middle  and  Late  epochs  (F,  (2. 
12)  =  4.8.  P  <  0.05)  but  not  between  Early  and 
Middle  epochs.  Reaction  time  did  not  differ  across 


Early  (543  -»•/— 169  msec).  Middle  (584  -r/  — 
155  msec),  and  Late  (593  +/—  97  msec)  epochs. 
The  increase  in  the  proportion  of  responses  dunng 
No- Press  trials  across  the  3  epochs  (10%,  16%,  and 
32%)  was  significant  ( F  (2,  12)  =  7.4.  P  <  0.01). 

Subjective  reports  were  consistent  with  the 
trends  in  the  behavioral  data  (Table  II).  Using  a 
7-levei  subjective  fatigue  scale  (1  indicates  maxi¬ 
mal  alertness  and  7  complete  exhaustion),  the  5 
subjects  reported  mean  ratings  of  3.42  +/  -  0.75 
dunng  the  Early  epoch.  5.05  +/—  1.23  during 
the  Middle,  and  6.22  —  /  —  0.44  during  the  Late 
(  F  (2.  12)  =  13.2.  P  <  0.001).  There  was  little  cor¬ 
relation  between  the  subjective  fatigue  ratings  and 
response  error  (bootstrap  correlation  =  —0.17 
+  /-  0.25). 

ERC  analysts:  prestimulus  preparatory  interval 

ERCs  from  the  prestimulus  interval  declined  in 
magnitude  ( F  (2.  243)  =  22.8,  P  <  10“’)  and  dif¬ 
fered  in  pattern  configuration  across  the  3  epochs 
(Fig.  3B  and  Table  I).  The  greater  change  from 
Early  to  Middle  epochs  was  revealed  by  the  low 


TABLE  1 


Significance  of  comparisons  across  Early,  Middle,  and  Late  epochs  of  the  day  for  each  of  3  neuroelectnc  measures  in  prestimulus, 
stimulus  processing,  and  response  inhibition  intervals  of  the  msuomotor  memory  task.  For  ERP  amplitude  and  ERC  magnitude,  the 
Early  vs.  Middle  vs.  Late  comparisons  were  performed  by  analysis  of  variance,  whereas  the  pairwise  comparisons  were  performed  by 
the  Scheffe  post-hoc  method.  Values  are  probabilities  (P  <  )  except  in  columns  heafpd  by  ‘ERC  pattern'  which  are  correlauons 
between  event-related  covariance  distnbutions.  The  standard  deviation  of  correlation  is  shown  in  parentheses. 


Interval 

Early  vs.  Middle  vs.  Late  epochs 

Early  vs.  Middle  epochs 

(msec) 

ERP 

ERC 

ERP 

ERC 

ERC 

amp. 

magnitude 

amp. 

magnitude 

pattern 

Presumulus 

(  -  562  to  -  62  msec) 

NS 

10'9 

NS 

io'9 

0.06  (0.09) 

Stimulus  processing 

(32-219  msec) 

NS 

0.01 

NS 

0.05 

0.71  (0.09) 

Response  inhibition 

(232  -469  msec) 

0.05 

•clO'10 

NS 

10'* 

0.67  (0.09) 

Early  vs.  Late  epochs 

Middle  vs. 

Late  epochs 

£R.C 

ERP 

ERc 

ERC 

ERP 

ERC 

ERC 

amp. 

magnitude 

pattern 

amp. 

magnitude 

pattern 

Prijtimulus 

7®  -62  (! 62  to  -62  msec) 

NS 

10'* 

0.01  (0.09) 

NS 

NS 

0.42(0.11) 

Stimulus  processing 

(32-219  msec) 

NS 

NS 

0.62  (0.07) 

NS 

0.05 

0.44(0.14) 

Response  inhibiuon 

(232-469  msec) 

0.05 

•«  10~ 10 

0.66  (0.07) 

NS 

0.0005 

0.51  (0.16) 
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TABLE  II 


TABLE  III 


Mean  subjective  fatigue  ratings  from  each  of  the  3  epochs  for 
the  5  subjects  based  on  a  7-level  subjecuve  fatigue  scale  (1 
indicates  maximal  alertness  and  7  complete  exhaustion). 


Subject 

Early 

Middle 

Late 

1 

3.51 

5.00 

6.12 

T 

4.12 

6.00 

6.00 

3 

3.20 

5.25 

6.00 

4 

4.00 

6.00 

7.00 

5 

2.25 

3.00 

6.00 

Mean 

3.42 

5.05 

6.22 

(S.D.) 

(0.75) 

(1.23) 

(0.44) 

Early- to  Middle  bootstrap  correlation  (0.06)  when 
compared  to  the  Middle-toLate  one  (0.42),  and 
by  the  significant  decline  in  Early-toMiddle  ERC 
magnitude  (F  (2,  243)  =  11.2,  P  <  0.0001)  when 
compared  to  the  non-significant  Middle-toLate 
difference.  Dominance  by  the  midline  central  (Cz) 
site  in  the  Early  epoch  was  replaced  by  right-sided 
dominance  (focused  at  the  right  anterocentral 
(aC4)  site)  in  the  Middle  epoch,  which  continued 
into  the  Late  epoch  (Table  III).  The  ERP  ampli¬ 
tude  in  this  interval  declined  by  15%  from  Early  to 
Middle  epochs  and  then  rose  by  3%  in  the  Late 
epoch:  the  change  over  the  3  epochs  was  not 
significant. 


ERC  analysis :  visual  stimulus  processing  interval 
The  amplitude  of  the  poststimulus  N1-P2  ERP 
component  did  not  change  with  time-on-task  (Fig. 
4A  and  Table  I,  middle  row).  The  mean  absolute 
ERC  magnitude  declined  from  the  Early  (9.5 


Sums  of  event-related  covariances  at  top  3  sites  in  prestimulus, 
stimulus  processing,  and  response  inhibition  intervals  for  Early. 
Middle  and  Late  epochs  of  the  day.  Designation  of  sites  is  as 
follows:  aC3  -  left  antero-centraJ.  aCz  -  midline  antero- 
centrai.  aC4  -  right  antero-centraL  C3  —  left  central.  Cz  - 
midline  central.  Pz  -  midline  parietal.  P4  —  nght  panetal.  pPl 
—  left  postero-panetal,  pP2  -  nght  postero-panetal. 


Interval 

Early 

Middle 

Late 

Presumulus 

i 

Cz 

624 

aC4 

2S2 

aC4 

235 

aCz 

415 

P4 

277 

P4 

221 

3 

ppi 

330 

aCz 

240 

C3 

162 

Stimulus  processing 

1 

Pz 

237 

Pz 

178 

Cz 

197 

Z 

Cz 

200 

Cz 

140 

pP2 

196 

3 

aCz 

173 

aC3 

123 

Pz 

187 

Response  inhibition 

i 

aCz 

574 

aCz 

284 

aCz 

103 

aC3 

360 

aC3 

199 

Cz 

77 

3 

aC4 

344 

aC4 

186 

aC4 

30 

+  /-  5.8)  to  Middle  (7.5  +/—  4.5)  epochs,  but 
then  rose  in  the  Late  (9.9  +/—  4.8)  epoch.  Al¬ 
though  this  change  across  the  3  epochs  was  signifi¬ 
cant  (F  (2.225)  =  4.8.  Pc  0.01),  the  effect  was  (2 ,-Zc~ 

considerably  less  robust  than  for  either  the  pre¬ 
stimulus  CNV  or  subsequent  P3  intervals.  The 
Early  and  Middle  ERC  patterns  (Fig.  4B)  were 
similar  (bootstrap  correlation  =  0.71  +  /  -  0.09), 
and  both  had  many  features  in  common  with  the 
Late  pattern.  In  particular,  the  midline  parietal 
(Pz)  and  midline  central  (Cz)  sites  were  among  the 
top  3  focal  sites  in  all  3  epochs  (Table  III).  The 
Middle  and  Late  patterns  differed  most  (bootstrap 
correlation  =  0.44  y—  0.14),  largely  because  in-  -re¬ 
volvement  of  the  right  postero-parietal  (pP2)  site 
increased  in  the  Late  pattern. 
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Fig.  3.  A^£rant?average  (over  all  5  subjects),  everfrelated  potential  (ERP)  laplacian  wave  forms  of  the  midline  central  (Cz)  electrode  A. 

during  a  700  msec  interval  when  subjects  were  expecting  the  next  VMMT  stimulus.  Accurate  Early  (594  trials,  solid  line).  Middle  (517 
trials,  dashed  line),  and  Late  epoch  (394  trials,  dotted  line)  averages  are  shown.  The  indicated  prestimulus  ERC  interval  was  used  for 
computing  the  event-related  covariance  patterns  shown  in  B.  The  root- mean- square  ERP  amplitude  of  this  interval  did  not 
significantly  differ  across  Early.  Middle,  and  Late  epochs.  B:  Event-related  covanance  (ERC)  patterns  characterizing  the  prestimulus 
Early  (left).  Middle  (center),  and  Late  (nght)  epochs.  The  view  is  from  above  the  head  with  the  nose  at  the  top  of  the  figure.  ERCs 
were  computed  from  ERPs  across  the  -562  to  -62  msec  prestimulus  interval  shown  in  A.  The  thickness  of  a  line  is  proportional  to 
the  negauve  loganthm  of  the  significance  (from  0.000005  to  0.05)  of  the  covanance  between  2  electrodes.  A  violet  line  indicates  a 
positive  covanance  (waves  of  same  polarity),  while  a  blue  line  represents  a  negative  covanance  (waves  of  opposite  polanty).  The 
radius  of  the  disk  at  each  site  is  proportional  to  the  sum  of  the  absolute  values  of  the  significant  ERCs  in  which  that  site  participated. 

Not  only  did  the  ERC  magnitude  greatly  decline  from  Early  to  Middle  to  Late  epochs,  but  the  ERC  pattern  also  changed,  with  the 
emphasis  shifting  from  the  midline  central  and  precentral  and  left  panetal  sues  to  the  nght  antero-central  and  panetal  and  left 

central  sites. 
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A; 

Fig.  4.*accurate  Earlv  (562  trials.  solid  line).  Middle  (517  trials.  dashed  line),  and  Late  (384  trials.  dotted  line)  grand-average  ERPs  of 
the  midline  mid-panetai  (Pz)  electrode  site,  showing  the  NI-P2  interval  followmg  presentation  of  the  visual  stimulus.  As  m  the 
presumulus  interval,  the  ERP  amplitude  did  not  differ  significantly  across  the  Early.  Middle,  and  Late  epochs.  B:  ERC  patterns 
characterizing  the  poststimulus  N1-P2  interval  in  Early  (left).  Middle  (center),  and  Late  (right)  epochs.  ERCs  were  computed  from 
ERPs  across  the  187  msec  long  poststimulus  interval  shown  in  A.  The  mean  ERC  magnitude  changed  only  moderately  from  Earlv  io 

Middle  to  Late  epochs;  the  top  focal  sues  remained  nearly  the  same. 
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AcCO^  Fig.  5.  A:  accurate  Early  (243  trials,  solid  line).  Middle  (250  trials,  dashed  line),  and  Late  (261  tnais.  dotted  line)  grand-average  ERPs 

for  the  finger  response  inhibition  (‘ No-Press)  trials,  showing  the  P3  peak  at  about  380  msec  after  ihe  stimulus  at  the  midline  ^ 
amero-cemrai  (aCz)  electrode  sue.  The  P3  amplitude  declined  from  Early  to  Middle  to  Late  epochs.  B:  '  N o^Press'  ERC  patterns  \]n  ' rfP.Z \ 

characterizing  Early  (left).  Middle  (center),  and  Late  (right)  epochs.  ERCs  were  computed  from  ERPs  during  the  281-468  msec 
poststimulus  interval  shown  in  A.  As  in  the  prestimulus  IntervaL  there  was  a  marked  global  decline  in  ERC  magnitude  However. 

V  unlike  the  case  in  the  prestimulus  interval  (but  like  the  N1-P2  interval),  the  3  patterns  were  very  similar. 
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ERC  analysis:  response-inhibition  interval 

ERCs  from  the  postsumulus  interval  of  No- 
Press  trials  declined  greatly  in  magnitude  (  F  (2. 
231)  =  54.7,  P  «  10“ 10),  but  did  not  differ  greatly 
in  pattern  configuration  (Table  I  and  Fig.  5B). 
There  was  also  a  significant  decline  in  P3  ampli¬ 
tude  (F  (2,  51)  =4.4,  P  <0.05).  The  stability  of 
the  pattern  configuration  was  seen  in  the  similar¬ 
ity  between  the  Early-to-Vliddle  and  the  Middle- 
to-Late  bootstrap  correlations,  and  by  the  domi¬ 
nance  of  the  patterns  by  the  antero-central  sites  in 
all  3  epochs  (Table  III).  There  were  highly  signifi¬ 
cant  ERCs  between  these  sites  and  the  midline 
parietal  (Pz)  and  right  posterior  parietal  (pP2) 
sites. 

Individual  subject  analysis 

We  next  determined  the  extent  to  which  indi¬ 
vidual  subject  patterns  corresponded  to  the  group 
patterns,  and  the  extent  to  which  it  was  possible  to 
distinguish  individual  trials  taken  from  different 
epochs  of  the  session.  Since  the  ERC  patterns 
from  the  prestimulus  intervals  in  the  Early  and 
Middle  epochs  were  so  strikingly  different,  we 
tested  inter-subject  variability  on  these  data  sets. 
Five  equations  were  formed  on  four-fifths  of  the 
trials,  and  tested  on  the  remaining  one-fifth. 

Discrimination  was  above  57%  (P  <  0.05)  for  3 
subjects,  but  was  49%  (chance)  for  the  other  two. 
Average  test-set  discrimination  was  62%  ( P  < 
0.001)  for  separate  analyses  of  the  first  3  subjects 
and  61%  ( P  <  0.001)  for  the  latter  two.  Finally,  a 
fourth  analysis  used  only  trials  from  the  subject 
with  the  best  classification  in  the  5-  and  3-subject 
analyses.  The  average  test-set  discrimination  was 
81%  ( P  <  0.001).  Thus,  individual  trials  from  the 
Early  and  Middle  data  sets  could  be  distinguished 
with  increased  accuracy  when  the  5  subjects  were 
divided  into  2  groups,  and  for  an  individual  sub¬ 
ject. 

Discussion 

A  fundamental  theme  in  contemporary  cogni¬ 
tive  neuroscience  is  that  mental  function  is  com¬ 
posed  of  multiple  information-processing  subsys¬ 
tems  associated  with  neural  networks  in  the  brain. 


The  British  neurologist  Henry  Head,  an  early  20th 
century  proponent  of  this  idea,  considered  that 
impaired  ‘vigilance’  could  cause  ‘high-grade  func¬ 
tions  (to)  suffer  in  general  or  in  part’,  and  that 
neural  systems  could  thus  be  affected  either  glob¬ 
ally  or  selectively  (Head  1926).  We  observed  2 
major  effects  of  sustained  mental  work  that  sup¬ 
port  Head’s  original  ideas:  (1)  a  global  decrease  in 
ERC  magnitudes,  consistent  with  the  well-known 
spatially  widespread  reduction  in  neuroelectnc 
signals  with  decreasing  alertness  (Gevuis  et  al. 
1977;  Broughton  1982:  Santamana  and  Chiappa 
1987);  and  (2)  a  previously  unreported,  temporally 
specific  change  in  ERC  pattern  configuration. 

The  prestimulus  ERCs  underwent  a  consider¬ 
able  change  in  pattern,  together  with  a  large  re¬ 
duction  in  magnitude,  from  the  Early  to  Middle 
epochs.  The  poststimulus  response  inhibition  ERC 
pattern  strength  was  also  reduced,  but  without 
much  pattern  change  from  Early  to  Middle  and 
from  Middle  to  Late  epochs.  Except  for  the  emer¬ 
gence  of  the  right  postero-parietal  site  in  the  Late 
epoch,  the  stimulus  processing  ERCs  underwent 
relatively  little  change  in  ERC  pattern  or  magni¬ 
tude.  These  results  suggest  that  specific  cortical 
areas  involved  in  response  preparation  were  most 
affected  by  prolonged  mental  work  and  that  those 
involved  in  response  inhibition  were  also  affected. 
Cortical  areas  involved  in  primary  visual  stimulus 
processing  were  considerably  less  affected. 

Although  the  source  of  ERCs  is  still  unknown, 
the  present  results  conform  to  the  hypothesis  that 
ERC  patterns  reflect  underlying  cortical  networks. 
The  prominence  of  the  midline  central  and 
antero-central  sites  in  the  prestimulus  interval  is 
consistent  with  a  cortical  network  component  re¬ 
sponsible  for  motor  preparation,  since  these  sites 
overlie  the  supplementary  motor  area  (Brodmann 
area  6).  The  declining  prominence  of  these  sites 
over  the  3  epochs  may  reflect  decreasing  involve¬ 
ment  of  the  motor  planning  component  with  sus¬ 
tained  mental  work,  perhaps  due  to  automatiza¬ 
tion.  That  these  sites  were  also  pronounced  in  the 
ERC  pattern  of  the  poststimulus  N1-P2  interval 
suggests  that  this  component  remains  in  operation 
following  the  stimulus.  Prominence  of  the  midline 
parietal  site  may  reflect  a  cortical  network  compo¬ 
nent  concurrently  involved  in  extracting  the  value 


11 


irve. 


of  the  stimulus  from  the  visual  image.  In  the 
poststimulus  P3  interval,  when  subjects  were  in¬ 
hibiting  their  response,  the  three  antero-central 
sites  largely  dominated  the  patterns  in  all  three 
epochs.  These  sites  overlie  supplementary  and  pre¬ 
motor  areas,  known  to  be  necessary  for  higher 
order  motor  control  such  as  response  inhibition. 
The  continued  prominence  of  these  sites  in  all  3 
epochs  suggests  that  these  areas  are  critically  in¬ 
volved  in  response  inhibition,  even  as  the  strength 
of  the  pattern  is  reduced  with  sustained  mental 
work. 

Our  findings  also  indicate  that,  since  dramatic 
changes  in  brain  activity  patterns  occurred  before 
performance  deteriorated  significantly,  measures 
of  brain  activity  may  be  more  sensitive  indicators 
of  the  deleterious  effects  of  sustained  mental  work 
than  measures  of  overt  behavior.  This  evidence 
could  explain,  in  part,  the  increased  vulnerability 
to  life  threatening  accidents  that  exists  during  the 
early  stages  of  prolonged  mental  activity,  when 
performance  has  not  yet  become  significantly  de¬ 
graded.  Further  refinement  of  the  techniques  used 
in  this  study,  and  their  application  to  single-trial 
data,  could  lead  to  the  development  of  on^-line 
warning  systems  in  which  persons  engaged  in  criti¬ 
cal  or  hazardous  work  would  be  alerted  to  the 
possibility  of  impaired  performance  due  to  mental 
overwork. 
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m.  FUNCTIONAL  TOPOGRAPHY  OF  LANGUAGE 


A.  Introduction 

This  study  was  designed  to  measure  spatiotemporal  evoked  potential  patterns  during  several  types  of 
linguistic  operations.  The  well-substantiated  role  of  the  ’language’  areas  of  posterior  frontal  and  tem¬ 
poral  cortices  of  the  dominant  hemisphere  makes  language  a  good  choice  for  studying  the  localization 
of  neurocognitive  processes.  Numerous  studies  have  attempted  to  measure  language-associated  lateral¬ 
ization  and/or  localization  in  the  EEG  and  EP,  but  to  date  the  findings  have  been  highly  inconsistant 
(reviewed  in  Beaumont,  1983;  Gevins,  1983;  Molfese,  1983;  Rugg,  1983;  Butler  and  Glass;  1986;  Gevins 
&  Cutillo,  1986;  Kutas  and  Van  Petten,  1988).  Processes  studied  include  acoustic  and  phonemic  pro¬ 
cessing  (Wood,  1975;  Molfese,  1979;  Lovrich  et  al.,  1986),  semantic  context  (Brown  et  al.,  1979,  Kutas, 
et  al.,  1984),  semantic  priming  (Rugg,  1985),  lexical  and  semantic  functions  (Buchsbaum  and  Fedio, 
1970;  Chapman  et  al.,  1978;  Fishier  et  al.,  1987),  and  syntax  (Kutas  and  Hillyard,  1980a,  1983).  The 
series  of  studies  by  Kutas  and  colleagues  include,  in  addition  to  the  N400  studies,  studies  of  natural 
sentence  processing,  grammar,  and  differences  between  open-class  (content)  and  closed-class  (function) 
words  (Kutas  and  Hillyard,  1980abc;  Kutas  et  al.,  1984;  Kutas  and  Vein  Petten,  1988).  Whereas  the 
many  studies  concentrated  on  the  EP  correlates  of  fairly  complex  linguistic  issues  (e.g.  "semantic 
appropriateness"),  we  directed  this  study  toward  the  measurement  of  the  spatial  and  temporal  charac¬ 
teristics  of  very  basic  linguistic  functions. 

A  number  of  studies  have  reported  language-associated  hemispheric  lateralization  of  EP  peaks,  some¬ 
times  with  larger  amplitudes  over  the  left  (dominant)  hemisphere.  Unfortunately,  many  of  these  stu¬ 
dies  were  methodologically  flawed  and  had  too  few  recording  channels  (review  in  Gevins  and  Cutilo, 
1986).  Regional  cerebral  blood-flow  studies  of  language  have  shown  patterns  of  activation  in  the  non¬ 
dominant  hemisphere  which  are  almost  mirror-images  of  those  in  the  dominant  hemisphere  (Lassen  et 
al.,  1978),  while  more  recent  PET  studies  have  reported  more  left-sided  activity  (Petersen  et  al., 
1988).  Further,  brain  stimulation  mapping  (Ojemann,  1983)  indicates  that  at  the  cortical  level, 
language  functions  are  organized  into  a  complex  "mosaic"  of  local  perisylvan  areas  on  the  order  of  a 
couple  centimeters  in  size,  which  are  involved  in  particular  aspects  of  language  processing.  Brain 
potentials  associated  with  bilateral  activation  might  not  exhibit  gross  lateralizations  of  amplitude,  and 
the  measurement  of  highly  localized  activity  would  require  more  detailed  spatial  sampling  than  is  usu¬ 
ally  used.  The  present  study  demonstrates  that  there  is  both  a  great  amount  of  commonality  between 
elementary  linguistic  processes,  as  well  as  highly  localized  differences  which  are  observable  through  the 
use  of  sufficiently  dense  spatial  sampling  and  techniques  of  spatial  enhancement  such  as  Laplacian 
Transform. 

Classical  neurological  and  neurolinguistic  thinking  has  emphasized  a  localizationist-serial  model  for 
language  processing  (see  for  example  Geschwind,  1970),  but  more  modern  models  consider  networks  of 
sub-processes  and  codes  (visual  or  auditory  encoding,  phonological,  semantic,  syntactic  processing) 
that  operate  in  a  parallel,  distributed  fashion  during  language  processing  (Petersen  et  al.,  1988;  Posner, 
1978).  In  a  recent  study,  Petersen  et  al.  (1988)  used  Oxygen-15  Positron  Emission  Tomography 
(PET)  to  study  healthy  subjects  performing  a  series  of  language  tasks.  The  activation  patterns  they 
found  were  generally  consistent  with  neurological  theory.  For  example,  activations  were  found  in  sup¬ 
plementary  motor  and  inferior  frontal  areas  for  word  generation,  and  in  cingulate  areas  for  attention. 
Their  results  also  suggest  that  phonological  decoding  for  familiar,  visually  presented  words  can  be  by¬ 
passed  by  good  readers,  consistent  with  current  cognitive-reading  theory.  A  somewhat  unexpected 
finding,  however,  was  the  activation  of  frontal  areas  during  the  lexical  processing  of  single  words, 
which  would  be  expected  to  involve  Wernicke’s  language  area  in  the  posterior  perisylvan  region. 
Although  their  results  were  interpreted  in  terms  of  currently  popular  parallel-processing  models,  this 
interpretation  must  be  considered  speculative  because  of  the  40  to  60-second  measurement  aperture  of 
the  0-15  PET  technique.  Without  more  precise  timing  information,  it  is  difficult  to  say  whether  areas 
were  active  during  the  entire  task  processing,  whether  they  were  sequentially  activated,  or  whether 
there  was  some  complex  combination  of  sequential  and  parallel  processes.  The  present  study  was 
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designed  to  obtain  temporal  detail  of  cortical  processing  during  receptive  language  processing:  form 
(or  letter)  perception,  phonemic  encoding,  semantic  retrieval,  and  simple  syntactic  judgements. 

In  our  experimental  design,  EEGs  were  measured  while  subjects  processed  sequential  pairs  of  visual 
stimuli  in  four-second  trials.  First,  a  cue  indicated  the  type  of  judgment  to  be  made,  and  then  the 
first  and  second  stimuli  (SI  and  S2)  were  presented  at  successive  1  sec  intervals.  The  subject  had  to 
determine  whether  the  S2  formed  a  'match'  to  the  Si.  To  form  a  'match,"  SI  and  S2  had  to  be 
physically  identical  in  the  graphic  condition,  be  homophones  in  the  phonemic  condition,  be  related  as 
antonyms  in  the  semantic  condition,  or  form  a  correct  subject-verb  combination  in  the  grammatical 
condition.  Subjects  responded  to  mismatches  with  a  left-hand  finger  press,  and  received  visual  feed¬ 
back  about  their  performance. 

There  are  conceptual  similarities  between  our  experiment  and  Petersen  et  al.’s  (1988)  paradigm, 
including  the  fact  that  neural  measurements  are  made  while  subjects  perform  increasingly  complex 
types  of  tasks.  Petersen,  et  al.,  assume  that  each  level  can  be  'subtracted"  from  the  next  "higher" 
level  in  order  to  isolate  distinct  higher-order  processes.  This  was  necessitated  in  part  by  the  40  to  60 
second  sampling  interval  required  by  the  0-15  PET  technique.  Our  experiment  differs  in  that  we 
were  also  able  to  obtain  information  about  sequential,  split-second  changes  in  the  patterns  of  activity 
of  cortical  networks  involved  in  graphic,  phonemic,  semantic  and  syntactic  processing.  (Our  technique 
does  not  however,  allow  full  three-dimensional  imaging  of  neural  activity.) 


B.  Hypotheses 

The  four  conditions  were  designed  to  differ  along  the  following  dimensions:  letter/non-letter  (graphic 
condition  versus  phonemic,  semantic  and  grammatic  conditions),  word/non-word  (graphic  and 
phonemic  versus  semantic  and  grammatic  conditions),  and  syntactic/non-syntactic  (grammatic  versus 
semantic  conditions).  Presumably,  sites  of  the  dominant  hemisphere  overlying  Broca’s  and  possibly 
Wernicke’s  areas  would  be  important  in  these  distinctions.  This  hypothesis  follows  from  the  findings 
of  Zurif  and  others  (Kolk,  1978;  Bradley  et  al.,  1980;  Zurif,  1980),  based  on  lesion  studies,  that  Broca’s 
area  is  involved  in  the  receptive  deficits  of  a  phonemic  nature.  Similarly,  semantic  processing  should 
be  distinguishable  from  phonemic  processing  at  sites  involving  Wernicke’s  area  for  the  processing  of 
meaning.  Grammatical  processing  should  be  distinguishable  from  semantic  processing  in  the  relative 
involvement  of  sites  over  and  anterior  to  Broca’s  area,  and  perhaps  Wernicke’s  language  area.  On  the 
basis  of  our  studies  of  task-specific  preparation  (Gevins  et  al.,  1987,  1989b),  we  expected  that  effects 
of  task-specific  preparatory  processes  might  be  seen  in  the  cue-to  stimulus  epoch. 


C.  Piloting 

There  were  two  phases  in  piloting  the  experiment.  In  the  first  phase,  50%  of  the  trials  were  matches, 
50%  were  mismatches,  and  subjects  were  required  to  make  a  motor  response  to  both.  Since  we  are 
particularly  interested  in  the  processing  associated  with  match  trials,  we  changed  the  probability  of  a 
match  to  85%,  and  had  subjects  respond  only  to  the  mismatch  trials  (15%).  This  eliminated  signals 
related  to  response  preparation  and  execution  from  the  match  trials,  while  the  15%  mismatch  trials 
served  primarily  to  ensure  that  the  subjects  were  performing  the  task.  To  help  ensure  subjects’  atten¬ 
tion  to  the  task,  the  rewards  were  changed  so  that  subjects  lost  25  cents  for  failing  to  respond  to  a 
mismatch. 
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D.  Methods 

1.  Subjects 

Nine  healthy,  right-handed  (modified  Edinburgh  Scale),  male  volunteers  between  the  ages  of  21  and 
35  participated  in  the  experiment.  None  had  any  history  of  neurologic  or  psychiatric  disease,  and  all 
were  native  English  speakers,  with  no  knowledge  of  Japanese. 

2.  Stimuli 

EEGs  were  recorded  while  subjects  determined  if  pairs  of  visual  stimuli  were  identical  in  appearance 
(graphic  condition),  were  the  same  in  pronunciation  (phonemic  condition),  had  opposite  meaning 
(semantic  condition),  or  formed  a  grammatically  correct  subject-verb  combination  (grammatical  con¬ 
dition)  (See  Figure  1  ).  Asterisks  were  positioned  in  front  of  and  after  each  stimulus  to  adjust  differ¬ 
ences  in  stimulus  length.  The  horizontal  visual  angle  was  under  two  degrees  and  the  vertical  visual 
angle  under  0.5  degrees.  The  average  luminance,  based  on  a  random  sampling  of  the  luminance  of 
240  stimuli  (60  from  each  condition),  was  1.43  lux. 

Graphic  condition: 

Stimuli  in  the  Graphic  condition  were  strings  of  Katakana  characters,  which  are  a  set  of  characters 
from  Kana,  the  syllabic  script  of  written  Japanese.  The  strings  were  either  three  (50%)  or  four  char¬ 
acters  (50%)  long.  Characters  that  were  highly  similar  to  each  other  or  that  had  disjointed  elements 
were  not  used.  In  graphic  match  trials  the  two  stimuli  were  identical.  In  mismatch  trials,  1  or  2  char¬ 
acters  of  S2  were  changed,  but  never  the  first  character.  The  number  of  characters  in  a  pair  was 
always  the  same. 

Phonemic  condition: 

Stimuli  in  the  phonemic  condition  were  pronounceable  non-words  which  conformed  to  the  rules  of 
English  orthography.  They  were  three  to  six  characters  in  length,  and  began  and  ended  with  one  to 
three  consonants.  Each  letter  of  the  string  contributed  to  its  pronunciation;  (e.g.,  the  consonant  com¬ 
bination  "ck,"  or  words  ending  in  a  consonant  followed  by  "e'  as  in  ‘dove’  were  not  used).  All  stimuli 
were  monosyllabic,  and  words  that  were  homophones  to  real  English  words  or  acronyms  were  not 
used. 

The  guidelines  discussed  by  Vitz  and  Spiegel- Winkler  (1973)  were  used  to  generate  the  non- words  and 
their  homophones  for  the  match  trials.  In  19%  of  the  pairs  the  first  consonant  or  consonantal  cluster 
was  changed;  in  40%  of  the  pairs,  the  final  consonant  or  consonantal  cluster  was  changed;  in  32%  of 
the  pairs,  the  vowel  or  vowel  cluster  was  changed,  and  in  5%  of  the  pairs  both  the  vowel  and  con¬ 
sonant  were  changed. 

Heterophonic  pairs  were  generated  by  changing  one  or  more  of  the  phonemes  of  S2  non-words.  In 
order  to  minimize  motor  cueing,  at  least  the  manner  or  place  of  articulation  of  the  original  phoneme 
and  the  phoneme  changed  in  the  S2  were  preserved.  For  example,  the  stop  consonants  /b/  or  /d/ 
were  foiled  by  other  stop  consonants  such  as  /p /  or  /t/,  but  never  by  the  nasal  or  fricative  consonants 
such  as  /m/  or  /f/.  Vowel  forms  that  have  more  them  one  pronunciation  (eg:  the  present  and  past- 
tense  forms  of  read  )  were  not  used.  In  3%  of  the  mismatch  pairs  the  first  consonemt  or  consonantal 
cluster  was  changed;  in  72%  of  the  pairs  the  final  consonant  or  consonantal  cluster  was  changed;  in 
6%  of  the  pairs  the  vowel  or  vowel  cluster  was  changed,  in  11%  of  the  pairs  both  the  vowel  or  con¬ 
sonant  or  vowel/consonantal  cluster  was  changed,  and  in  5%  of  the  pairs  both  the  first  and  final  con¬ 
sonants  were  changed.  For  example,  terp  and  tirb,  trolph  and  trorf. 
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Semantic  condition: 

The  stimuli  of  the  semantic  task  were  formed  from  open-class,  relatively  high-frequency  monosyllabic 
English  words.  Match  trials  were  formed  from  two  words  that  are  antonyms,  defined  as  two  words 
representing  a  concept  in  oppositional  relation  to  each  other  (Gardner  et  al.,  1976).  The  antonyms 
were  selected  from  a  variety  of  sources  including  Bolander  et  al.  (1985)  and  Urdang  (1984),  and  were 
always  of  the  same  functional  category  (verbs  with  verbs,  adjectives  with  adjectives).  75%  were  noun 
or  verb  pairs,  25%  adjective  or  adverb  pairs.  Words  with  irregular  pronounciation,  or  more  them  one 
pronunciation  were  not  used.  The  mismatch  pairs  were  composed  of  words  meeting  the  criteria 
described  above,  except  that  they  were  not  related  semantically,  phonetically  or  formally  (e.g.,  hot , 
real ). 

Grammatical  condition: 

Grammatical  stimulus  pairs  were  composed  of  a  pronoun  followed  by  a  verb  which  formed  a  short 
meaningful  sentence  (e.g.,  He  swims).  The  verbs  were  all  open  class,  high  frequency,  monosyllabic, 
and  intransitive  (Webster’s  New  Universal  Unabridged  Dictionary,  1983).  Verbs  were  used  as  many 
as  three  times,  but  always  with  a  different  agent.  As  in  the  semantic  condition,  words  with  irregular 
pronounciation  or  multiple  pronunciations  were  not  used.  Mismatched  pairs  were  composed  of  pro¬ 
noun  verb  combinations  which  did  not  form  a  complete,  grammatical  sentence.  The  S2  was  a  noun  or 
adjective  that  could  not  be  used  as  a  verb  (such  as:  he  oat  ),  but  the  S2  was  never  neologistic  (such  as 
forming  bolds  from  the  adjective  bold).  Moreover,  the  S2  verb  was  conjugated  incorrectly  with  respect 
to  the  agent,  so  that  the  third-person  conjugation  was  given  to  the  first-person  forms,  and  first-person 
conjugation  to  third-person  forms. 

3.  Types  of  Trials 

Each  triad  of  the  task  began  with  a  visual  cue  that  signaled  the  condition  of  that  trial  (L:  "looks 
alike";  S:  "sounds  alike";  O:  "opposites';  G:  "grammatic").  Conditions  were  presented  in  random 
order.  Si  was  presented  1  sec  later,  and  S2  1  sec  after  Si.  Subjects  were  instructed  to  press  a  button 
with  the  left  index  finger  for  mismatched  pairs  (15%  of  the  trials).  The  maximum  allowable  RT  on 
mismatch  trials  was  1500  msec.  Subjects  were  instructed  to  press  when  uncertain  whether  the  trial 
was  a  match  or  mismatch.  There  were  5  possible  types  of  feedback: 

(1)  Correct  match  trial  (no-press):  Fixation  dot  appears  1500  msec  after  S2.  The  subject  earned 
two  cents  for  not  responding. 

(2)  Correct  mismatch  trial  (press):  feedback  symbol  "+"  was  presented  one  sec  after  response 
onset.  The  subject  earned  2  cents  for  responding. 

(3)  False  positive  (incorrect  match  trial):  feedback  "0"  was  presented  750  msec  after  response.  The 
subject  earned  no  money. 

(4)  Incorrect  mismatch  trial  (no-press):  feedback  symbol  was  presented  1200  msec  after  S2. 
The  subject  lost  25  cents. 

(5)  A  downward  triangle  "V"  was  presented  immediately  after  a  response  that  was  produced  too 
soon  (less  than  300  msec)  after  S2.  The  subject  neither  earned  nor  lost  money. 

The  intertrial  interval  was  automatically  delayed  until  there  were  no  detectable  eye-movements,  and 
was  at  least  2  sec  long.  Subjects  were  given  80  training  trials  before  recording  began.  Duration  of 
cue,  Si,  S2  and  feedback  was  150  msec. 
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4.  Recordings 

Each  subject  performed  1200  task  trials  presented  in  blocks  of  20,  randomized  as  to  condition  and 
response  requirement.  Subjects  were  given  four  blocks  of  training  before  EEG  data  were  recorded. 
This  was  sufficient  in  most  cases  for  performance  to  reach  a  stable  level.  If  not,  a  second  set  of  80  tri¬ 
als  was  given.  Subjects  earned  $7.00/hour,  in  addition  to  the  bonuses  for  correct  performance. 

EEGs  were  reco",ed  from  59  scalp  electrodes  referenced  to  the  midline  anterior-parietal  (aPz)  elec¬ 
trode.  The  montage  was  an  extended  10-20  system  (Gevins,  1988),  and  included  the  frontal  sites  aFl 
and  aF2,  Fz,  F3  to  F8,  and  Fpz;  the  anterior  central  aCz,  aCl  to  aC6;  Central  Cz,  C3  &.  C4;  anterior 
temporal  aT5  &.  aT6;  temporal  T3  to  T8:  lower  temporal  1T1,  1T2,  1T5,  1T6;  ventral  temporal  vT5  and 
vT6;  anterior  parietal  aPl  to  aP6;  parietal  Pz,  P3  to  P6;  anterior  occipital  aOl  &.  a02;  occipital  Oz, 
Ol  &;  02;  ventral  occipital  vOl  &;  v02;  and  the  mica  (I)  and  both  mastoids  (Ml  &:  M2).  Vertical  eye 
movements  were  recorded  bipolar ly  from  an  electrode  pair  placed  supra-  and  suborbitally;  horizontal 
eye  movements  were  recorded  bipolarly  between  electrodes  at  the  outer  canthus  of  each  eye.  Other 
bipolar  pairs  were  placed  over  flexor  digitori  muscles  of  left  and  right  arms  to  record  EMG,  and  at  the 
submentalis  to  record  subvocal  movements  of  the  larynx  and  mouth.  The  EEG  was  amplified  8333 
times,  band-pass  filtered  from  0.5-50  Hz  and  recorded  at  128  samples  per  sec.  Electrode  positions 
were  digitized  in  3  dimensions  with  a  Balhemus  Navigation  Sciences  3-SPACE  ISOTRACK  in  order 
to  accurately  determine  interelectrode  distances  for  computation  of  a  2-dimensional  a  nearest-neighbor 
local  Laplacian  transform. 

5.  Analysis 

Trials  with  eye  movement,  left-hand  EMG  in  match  trials,  any  right-hand  EMG,  or  instrumental 
artifacts  were  eliminated  by  visual  editing  prior  to  averaging.  An  optimal,  least-squares  estimate  of 
the  Laplacian  operator  was  applied  to  the  single-trial  potential  timeseries  to  compute  the  Laplacian 
Derivation  (LD  -  see  section  IV.C).  This  reduces  the  high  degree  of  spatial  overlap  of  potentials  at 
the  scalp,  and  removes  the  effect  of  the  location  of  the  reference  electrode.  Peripheral  sites  for  which 
the  Laplacian  amplitude  cannot  be  accurate!,  calculated,  and  several  bad  amplifier  channels  (C3  and 
T7),  along  with  their  homologues  (C4  and  T8),  were  eliminated  from  the  list  of  channels  for  which  the 
Laplacian  Derivation  (LD)  was  calculated.  There  were  39  channels  remaining:  aFl,  aF2,  Fz,  F3  to 
F8,  Fpz;  aCz,  aCl  to  aC6;  Cz;  aPz,  aPl  to  aP6;  Pz,  P3  to  P6;  T3  to  T6;  aOl,  a02;  and  Ol,  02  and 
Oz  (Figure  2). 

Averages  were  formed  from  correctly  performed  match  and  mismatch  trials  for  each  condition,  and 
lowpass  filtered  at  7  Hz.  The  four  final  data  sets  contained  1762  graphic,  1688  phonemic,  1592  seman¬ 
tic,  and  1741  grammatic  match  trials.  A  digitally  linked  ears  reference  was  computed  to  examine 
EPs.  Since  a  large  amount  of  common  activity  was  evident  in  the  linked-ears  EPs,  we  will  mostly  dis¬ 
cuss  the  Laplacian  Derivation  results,  except  where  comparison  with  the  potentials  is  helpful  in  inter¬ 
preting  the  LD  waveforms. 

E.  Results 

1.  Behavioral 

For  the  infrequent  (15%)  mismatch  trials,  the  mean  reaction  times  were,  for  the  edited/sorted  (and 
unedited)  data:  graphic  condition  -  607  (605)  msec;  phonemic  -  720  (713)  msec;  semantic  -  847  (839) 
msec;  grammatic  -  737  (728)  msec.  Except  for  the  phonemic-grammatic  pair,  all  reaction  time  differ¬ 
ences  were  significant.  In  the  final  edited  data  sets,  however,  the  standard  deviations  were  small  and 
similar  across  subjects:  graphic  -  101  msec;  phonemic  -  117  msec;  semantic  -  131  msec;  grammatic  - 
126  msec.  Percentage  of  false  positives  (move  to  match)  and  missed  targets  (no-move  to  mismatch)  in 
the  unedited  data  were:  graphic,  5.6%/l2.9%;  phonemic,  5.9%/14.0%;  semantic,  12.8%/15.8%;  gram¬ 
matic,  3.8%/4.7%. 
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2.  Neurophysiological 

Laplacian  Derivation  (LD)  averages:  Differences  between  conditions  in  peak  latency  and  topography 
were  very  complex  in  the  linked-ears  EPs,  due  to  extensive  overlap  of  potential  fields  at  the  scalp. 
The  between-condition  differences  were  highly  simplified  in  the  LD  waveforms,  where  a  peak  is  fre¬ 
quently  seen  at  only  a  few  locations. 

The  conditions  were  divided  on  the  basis  of  task  requirements  into  letter/non-letter  conditions 
(phonemic,  semantic  and  grammatic  vs.  graphic),  word/non-word  conditions  (semantic  and  gram- 
matic  vs.  phonemic  and  graphic,  or  simply  semantic  vs.  phonemic),  and  syntactic/non-syntactic  con¬ 
ditions  (grammatic  vs.  semantic).  Hypotheses  were  tested  for  each  electrode  separately.  Figures  3,  4, 
and  5  show  the  9-person,  averaged  LD  waveforms  for  match  trials  for  the  cue,  first  stimulus,  and 
second  stimulus,  respectively. 

[NOTE:  In  labeling  peaks  in  the  LD  waveforms  a  'P'  is  used  to  denote  emerging  current  at  the  scalp, 
and  an  "N'  for  entering  current.  This  polarity  does  not  necessarily  correspond  with  the  positive  and 
negative  voltage  polarity  of  the  linked-ears  EP  peaks. 


Letter /Non-letter  Comparisons: 

A  large  number  of  between-condition  differences  were  found  between  the  letter  and  nonletter  condi¬ 
tions.  The  cue  for  the  graphic  condition  elicited  an  amplitude  increase  in  the  N323  peak,  mainly  at 
lateral  anterior  occipital  channels.  The  three  letter  conditions  all  had  smaller  peaks.  N323  was  maxi¬ 
mal  at  the  aOl  electrode,  where  the  negative  (entering  current)  peak  in  the  graphic  condition  was 
about  twice  as  large  as  the  language  conditions  (t(l,8)=2.63,  p<  .05).  Figure  3  shows  this  effect  on 
the  N323  peak  in  posterior  channels  in  the  nine-person  LD  averages.  The  effect  was  slightly  left- 
lateralized.  The  Contingent  Negative  Variation  (CNV)  preceding  Si  was  also  larger  for  the  graphic 
condition  at  left  hemisphere  channels,  including  aOl  (t(l,8)=2.28,  p<  .05).  For  both  the  N323  and 
the  CNV,  an  ANOVA  on  the  three  language  conditions  indicated  that  their  amplitudes  did  not  differ 
significantly  at  most  of  the  electrodes  where  the  graphic  condition  had  significantly  larger  amplitudes. 
Figure  6  [TOP  row]  shows  the  electrodes  whose  amplitude  significantly  differed  between  the  graphic 
and  the  language  conditions  presented  on  a  cartoon  of  a  lateral  view  of  the  head. 

Post-Si  condition  differences  related  to  the  letter/non-letter  dimension  were  apparent  in  the  averaged 
LD  waveforms  in  the  N250  peak  (Figure  4),  which  was  larger  in  the  graphic  condition.  The  effect  was 
largest  at  P5  (P5:  t(l,8)=6.15,  p<  .0005;  T5:  t(l,8)=4.12,  p<.005;  and  aOl:  t(l,8)=2.49,  p<.05).  At 
326  msec  after  Si  there  was  a  positive  peak  in  the  graphic  condition,  visible  only  at  right  frontal  sites 
(F8:  t(l,8)=5.16,  pc.OOl;  F6:  t=2.74,  p<.05).  At  568  msec  there  was  a  negative  peak  maximal  at  F4 
(t(l,8)=4.18,  pc.Ol).  Figure  6  [  MIDDLE  row]  shows  the  electrodes  whose  N250  and  N568  amplitude 
significantly  differed  between  the  graphic  and  the  language  conditions. 

S2  elicited  a  negative  wave  at  258  msec  at  posterior  sites,  which  is  similar  in  morphology  to  the  N250 
elicited  by  Si  (see  Figure  5).  It  was  likewise  larger  in  amplitude  in  the  graphic  condition,  the  effect 
again  being  most  significant  at  P5  (t(  1 ,8) =5 .32,  pc.OOl).  (An  inspection  of  the  waveforms  of  Figure  5 
might  suggest  that  the  graphic  condition  should  also  be  significantly  larger  at  the  P6  electrode.  This 
is  not  the  case,  however,  because  the  standard  deviation  of  N258  at  P6  is  twice  as  large  as  the  mean 
difference  between  conditions.  By  contrast,  the  standard  deviation  of  N258  at  P5  is  only  about  half 
the  mean  difference.  There  are  numerous  other  instances  in  these  data  where  an  apparently  large 
amplitude  difference  in  the  waveforms  does  not  reach  significance  due  to  variability  across  subjects.) 
An  ANOVA  on  the  three  language  conditions  indicated  that,  except  for  the  P5  and  P3  electrodes,  the 
language  conditions  did  not  differ  significantly  among  themselves.  At  460  msec  after  S2,  there  was  a 
peak  in  the  graphic  condition  that  reverses  polarity  between  anterior  central  and  frontal  midline  sites 
(aCz  and  Fz),  where  it  was  positive,  and  right  anterior  central  and  frontal  sites,  where  it  was  negative. 
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Its  amplitude  difference  from  language  conditions  was  greatest  at  right  frontal  sites  (F4:  t(l,8)=4.82, 
p<.005;  F6:  t(l,8)=4.40,  p<  .005),  and  at  the  left  occipital  site  01  (t(l,8)=4.95,  p<  .005).  These 
effects  are  illustrated  in  Figure  6  [BOTTOM  row]. 

In  the  above  instances  the  graphic  condition  had  larger  peaks  than  the  language  conditions.  There 
were  also  instances  in  which  the  language  conditions  had  larger  peaks;  in  some  cases,  the  waves  were 
entirely  absent  in  the  graphic  condition.  A  positive  peak  at  313  msec  after  Si  distinguished  the 
language  conditions  most  clearly  at  the  left  frontal  site  F5  (t(l,8)=2.63,  p<.05).  (Although  the  F3  site 
looks  like  it  should  differ  between  language  and  graphic  conditions,  the  effect  was  not  significant  due 
to  a  large  standard  deviation.)  A  large  negative  peak  occured  at  448  msec  in  the  language  conditions, 
most  prominent  at  T3,  aC5  and  aOl  (T3:  t(l,8)=5.45,  pc.OOl;  aC5:  t(l,8)=5.90,  p<  .001;  aOl: 
t(l,8)=5.45,  p<  .001).  An  ANOVA  on  the  3  language  conditions  indicated  that  the  three  language 
conditions  did  not  differ  in  amplitude  at  the  sites  where  they  differed  from  the  graphic  condition 
(except  for  aOl  and  aCl).  After  S2,  the  P279  peak  exhibited  both  language  and  graphic  effects:  it 
was  larger  at  left  frontal  sites  for  the  three  language  conditions  (t(l,8)=3.58,  pc.Ol  at  F3),  and  larger 
at  the  midline  frontal  site  (Fz:  t(l,8)=3.08,  p<  .05)  for  the  graphic  condition.  The  significantly  larger 
amplitudes  for  the  graphic  condition  at  left  posterior  sites  and  at  right  anterior  and  posterior  sites  are 
due  mainly  to  the  descending  limb  of  the  N460  peak,  which  is  significantly  larger  even  at  this  early 
latency.  Figure  7  shows  the  sites  where  language  processing  produced  larger  amplitudes  than  graphic 
processing  (except  the  above  mentioned  complex  of  activity  for  P279,  which  was  larger  in  the  graphic 
condition  at  Fz,  at  right  posterior,  and  at  left  hemisphere  sites). 

Word  thc/Nonword  Comparisons: 

Word/non-word  differences  were  tested  by  comparing  the  graphic  and  phonemic  conditions  combined 
versus  the  semantic  and  grammatic  conditions  combined,  and  also  by  simply  testing  the  phonemic 
versus  semantic  conditions.  Differences  between  word-stimulus  conditions  and  non-word-stimulus  con¬ 
ditions  occurred  late  in  the  epochs  following  Si  and  S2. 

Significant  differences  between  word  (semantic  and  grammatic)  and  non-word  conditions  (graphic  and 
phonemic)  were  maximal  at  about  684  msec  after  Si.  Electrodes  where  the  phonemic  condition  had 
larger  amplitudes  were  aC5,  F5,  F8,  Fz,  aP5,  T3  and  T4.  The  strongest  effects  were  at  T3 
(t(l,8)=3.74,  p<  .01)  and  aC5  (t(l,8)=4.12,  p<  .005).  In  the  case  of  the  P719  peak  following  Si,  the 
semantic  condition  had  larger  amplitudes  than  the  phomenic  at  Fz  (t(l,8)=4.33,  p<.005),  aP5 
(t(l,8)=2.88,  p<.05),  aOl  (t(l,8)=2.32,  p<  .05),  and  Ol  (t(l,8)=2.43,  p<  and  non-word  processing 
following  Si. 

Although  the  strongest  language  effects  in  this  study  tended  to  occur  over  the  left  hemisphere, 
word/non-word  differences  after  S2  occur  earlier  over  the  right  hemisphere  than  the  left.  The  mean 
difference  was  204  msec,  S.D.=87  msec  (p<.005).  This  distinction  of  word  and  non-word  conditions, 
or  simply  between  phonemic  and  semantic  conditions,  was  significant  at  right  hemisphere  sites  at 
about  365  msec,  in  a  rather  diffuse  group  of  electrodes  including  T4,  P4,  aP2,  aP4,  aP6,  F4  and  aF2. 
Differences  occur  over  the  left  hemisphere  at  a  later  latency  (about  563  msec).  The  broad  N563  wave, 
strongest  at  T3,  aC5  and  F5,  and  another  wave  of  opposite  polarity  at  aOl  and  a02,  distinguished  the 
phonemic  from  semantic  and  grammatic  conditions,  the  phomenic  condition  having  larger  amplitudes 
in  both  cases.  The  word/non-word  differences  at  365  and  563  msec  after  S2  are  illustrated  in  Figure  8 
[BOTTOM  row]. 

Syntactic /Non- syntactic  Comparisons: 

Syntactic  effects,  demonstrated  by  differences  between  the  grammatic  and  semantic  conditions,  occur 
after  both  Si  and  S2.  After  Si,  there  was  a  large  negative  peak  at  442  msec  at  left  fronted  (F3  and 
F5)  and  anterior  central  (aCl)  sites,  which  was  larger  in  the  grammatic  than  semantic  condition  (F3: 
t(l,8)=2.40,  p<.05;  F5:  t(l,8)=2.52,  p<.05;  aCl:  t(l,8)=3.07,  p<  .05).  It  appears  to  be  an  anterior 
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extension  of  the  N448  peak  which  occurs  at  T3  in  all  language  conditions  (see  Figure  4).  However, 
examination  of  the  linked-ears  EP  waveforms  indicates  that  the  picture  was  more  complicated.  In  the 
aCz  channel,  where  the  EP  peaks  were  largest,  the  semantic  and  phomenic  peaks  were  negative  in 
potential,  but  the  grammatic  peak  was  positive.  There  were  even  stronger  differences  at  several  poste¬ 
rior  right-hemisphere  sites  including  P4,  P6,  and  T6  (p<.001),  an  effect  due  to  a  positive  wave  peak¬ 
ing  at  about  450-500  msec.  It  is  not  clear  how  this  posterior  right-sided  positive  activity  is  related  to 
the  N448  and  N442  waves  at  left  frontal  sites.  The  effects  at  aC6  (t(l,8)=4.96,  p<  .005)  and  F6 
(t(l,8)=2.38,  p<  .05)  differ  from  the  other  gramatic/semantic  comparisons  in  that  the  semantic  condi¬ 
tion  has  larger  amplitudes  at  these  sites,  with  a  polarity  reversal,  being  negative  at  aC6  and  positive 
at  F6. 

After  S2,  the  grammatic  condition  was  distinguished  from  the  semantic  condition  by  a  larger  ampli¬ 
tude  of  the  P279  peak  at  midline  and  left  frontal  electrodes  Fz  (t(l,8)=3.58,  p<  .01),  F5  (t— 4.49, 
p<.005),  and  F3  (t=3.18,  p<.05),  and  also  at  the  left  anterior  parietal  electrode  aP3  (t(l,8)=3.63,  p< 
.01),  and  the  right  anterior  central  site  aC2  (t(l,8)=3.55.  p<  .01).  A  number  of  other  electrodes 
reached  significance  at  p<  .05.  The  locations  of  the  syntactic  effects  for  N442  and  P279  are  shown  in 
Figure  9.  Finally,  the  N460  peak  after  S2  distinguished  the  grammatic  condition  at  midline  frontal 
sites  Fz  (t(l,8)=3.45,  p<  .01)  and  Fpz  (t=3.46;  p<  .01),  and  at  F3,  P3,  and  T5  (all  p<  .05).  (N460 
not  included  in  illustrations.) 
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LANGUAGE  MONTAGE 


Figure  1.  Examples  of  cues  and  stimuli,  and  their  timing,  for  the  graphic,  phonemic,  semantic  and 
grammatic  conditions.  For  each  condition,  the  subject  had  to  judge  whether  the  second  stimulus  (S2) 
matched  the  first  stimulus  (Si). 
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LANGUAGE  TASKS 
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Figure  2.  Cartoon  showing  the  locations  of  the  39  non-peripheral  Laplacian  Derivation  sites. 
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Figure  3.  Cue-evoked  Laplacian  Derivation  (LD)  waveforms  for  eleven  posterior  channels  averaged 
over  all  9  subjects.  The  N323  peak  and  subsequent  CNV  were  larger  in  the  graphic  condition  over 
visual  association  cortex  when  subjects  were  cued  to  prepare  for  processing  unfamiliar  Japanese  kata- 
kana  characters.  Waveforms  were  lowpass  filtered  at  7  Hz.  The  x-axis  shows  one  second  from  Cue  to 
the  first  stimulus.  The  y-axis  corresponds  to  +/-0.18  microvolts  per  square  centimeter.  The 
waveforms  for  the  4  conditions  are  shown  in  the  same  colors  in  all  illustrations:  graphic  [black] , 
phonemic  [blue),  semantic  [green],  grammatic  [red]. 
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Figure  4.  LD  waveforms  evoked  by  the  first  stimulus,  averaged  over  9  subjects  and  lowpass  filtered  at 
7  Hz,  The  y-axis  corresponds  to  +/-0.216  microvolts  per  square  centimeter.  Graphic  [black], 
phonemic  [blue],  semantic  [green],  grammatic  [red]. 
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Figure  5.  LD  waveforms  evoked  by  the  second  stimulus,  averaged  over  9  subjects  and  lowpass  filtered 
at  7  Hz.  The  x-axis  shows  one  second  beginning  with  S2.  The  y-axis  corresponds  to  +/-0.234  micro¬ 
volts  per  square  centimeter.  Graphic  [black],  phonemic  [blue],  semantic  [green],  grammatic  [red]. 
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Figure  6.  [TOP  row]  Left  and  right  hemisphere  electrode  sites  at  which  preparation  for  the  graphic 
stimulus  had  a  larger  post-cue  N323  peak  and  CNV  than  the  for  the  three  language  conditions.  [MID¬ 
DLE  row]  Sites  where  the  first  stimulus  of  the  graphic  condition  evoked  greater  N250  and  N568  peak 
amplitudes  than  language  conditions.  [BOTTOM  row]  Sites  where  the  second  stimulus  and  subse¬ 
quent  matching  processes  elicited  larger  N258  and  N460  peaks  in  the  graphic  condition.  [Significance 
level  indicated  by  size  and  shading  of  dots;  see  Figure  2  for  electrode  names.] 
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Figure  7.  [TOP  two  rows]  Sites  where,  after  the  first  stimulus,  the  language  conditions  produced 
greater  P313  and  N448  peak  amplitudes  than  the  graphic  condition.  [BOTTOM  row].  Sites  where 
the  S2-evoked  P279  peak  had  differences  between  language  and  non-language  conditions.  [Significance 
level  indicated  by  size  and  shading  of  dots;  see  Figure  2  for  electrode  names.] 
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Figure  8.  [TOP  row]  Word/non-word  differences  foUowing  the  first  stimulus:  sites  where  the  N684 
peak  was  larger  for  phonemic  stimuli  (pronouncable  non- words)  compared  to  semantic  stimuli  (words), 
and  where  the  semantic  stimuli  produced  larger  amplitudes  than  the  phonemic  for  the  P719  peak. 
[BOTTOM  row]  Following  the  second  stimulus,  the  semantic  and  phonemic  conditions  differ  over  the 
right  hemisphere  by  365  msec,  and  considerably  later,  at  563  msec,  over  the  left  hemisphere.  [Signifi¬ 
cance  level  indicated  by  size  and  shading  of  dots;  see  Figure  2  for  electrode  names.] 
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Syntactic  Processes 
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Figure  9.  Sites  of  amplitude  increases  in  the  grammatic  condition  compared  to  the  semantic  condition 
for  the  N442  peak  following  SI  [TOP],  and  the  P279  peak  following  S2  [BOTTOM],  [Significance  level 
indicated  by  size  and  shading  of  dots;  see  Figure  2  for  electrode  names.] 
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F.  Discussion 

1.  Letter  (phonemic,  semantic  and  grammatic  conditions)  versus 

Non-letter  (graphic  condition)  Comparison 

Differences  corresponding  to  the  letter/non-letter  distinction  were  the  most  numerous,  the  first  differ¬ 
ence  being  elicited  by  the  task  cue  preceding  the  first  stimulus.  The  augmentation  of  the  N323  peak 
at  occipital  and  posterior  temporal  electrodes  in  the  graphic  condition  occured  in  response  to  the 
task-specific  cue  letter  "L",  and  not  for  cue-letters  signaling  one  of  the  language  conditions  ("S",  "O" 
or  "G").  Since  all  four  cues  were  alike  in  gross  physical  characteristics,  such  as  size,  location,  lumi¬ 
nance  and  duration,  and  since  conditions  were  randomized,  the  increase  in  N323  may  reflect  a 
preparatory  process  involving  the  preparation  of  visual  association  (and  inferotemporal)  cortex  for  the 
processing  of  the  unfamiliar  katakana  characters.  Although  the  percentage  of  cues  for  the  graphic 
condition  was  25%,  and  thus  rarer  than  cues  for  the  language  conditions,  the  distribution  of  P323  in 
the  linked-ears  EPs  was  not  that  of  the  "classic"  oddball  P300:  it  was  left-iateralized,  with  positive 
maximum  at  aOl,  and  negative  minima  at  central  and  frontal  sites  (Figure  10,  LEFT).  Further,  the 
distribution  of  the  P300  to  the  S2  of  mismatch  trials  (not  reported  here)  did  have  the  "typical"  P300 
distribution  for  these  subjects.  Thus  we  may  conclude  that  the  P323  (and  its  Laplacian  Derivation 
equivalent  N323)  was  not  simply  a  P300  produced  by  the  relatively  lower  probability  of  graphic  condi¬ 
tion  cues,  and  that  its  amplitude  increase  may  be  evidence  for  a  greater  priming  of  visual  association 
(and  inferotemporal)  cortex  in  anticipation  of  the  non-familiar  visual  stimuli,  as  compared  to  that 
needed  for  the  highly  familiar  letters.  Although  such  priming  has  been  suggested  by  psychological 
studies,  the  result  reported  here  presents  direct  neural  evidence  of  this  phenomena. 

If  the  N323  effect  does  reflect  some  aspect  of  prestimulus  priming  of  cortical  areas  involved  in  visual 
encoding,  it  would  be  reasonable  to  expect  an  increase  in  the  Contingent  Negative  Variation  (CNV)  at 
electrodes  overlying  these  cortical  areas.  This  was  indeed  the  case.  The  late  cue  epoch  contained  a 
negative  shift  (positive  in  the  LD  waveforms  -  Figure  3),  mainly  over  left  posterior  and  anterocentral 
sites;  of  these  sites,  aOl  and  T5  also  had  an  amplitude  increase  in  the  N323.  Finally,  at  most  of  the 
significant  sites,  the  ANOVA  indicated  that  the  three  language  conditions  did  not  differ  among  them¬ 
selves.  This  was  also  true  for  the  N250  and  N258  peaks  elicited  by  the  first  and  second  stimuli.  This 
suggests  that  proceses  reflected  in  these  peaks  are  affected  purely  by  the  letter/non-leter  difference. 
In  the  case  of  the  Cue-evoked  N323,  this  process  may  be  priming  of  visual  association  cortex,  while  for 
the  Sl-evoked  N250  and  S2-evoked  N258  it  may  be  the  actual  encoding  or  analysis  of  higher-order 
physical  features. 

The  next  difference  between  graphic  and  letter  stimuli  occured  in  the  N250  peak  elicited  by  the  first 
stimulus.  Although  all  four  conditions  elicited  an  N250,  it  was  significantly  larger  for  graphic  stimuli 
at  the  left  lateral  parietal  (P5)  and  other  nearby  sites.  This  effect  may  reflect  the  greater  amount  of 
processing  required  to  encode  the  unfamiliar  graphic  figures.  It  is  not  surprising  that  the  stimuli  used 
in  the  graphic  condition  produce  more  activity  over  the  left,  rather  than  right  posterior  hemisphere. 
Studies  in  primates  (Ungerleider  and  Mishkin,  1982)  and  humans  (Kosslyn,  1988)  suggest  that  the 
encoding  (and  imaging)  of  complex  visual  figures  occurs  primarily  in  areas  of  left  posterior  cortex. 
The  electrode  sites  at  which  the  graphic  condition  produced  significantly  greater  activity  (aOl  for 
N323,  P5  and  neighboring  electrodes  for  N250  following  Si  and  P258  folowing  S2)  are  quite  consistent 
with  the  above  authors’  proposed  ventral  system  for  visual  encoding.  This  system  extends  via 
cortico-corticaJ  connections  from  the  visual  cortex,  through  the  left  tempero-parieto-occipital  region,  to 
the  inferior  temporal  cortex.  After  the  second  stimulus,  the  graphic  condition  again  elicited  larger 
negative  peaks  (N258)  mainly  at  left  posterior  parietal  and  anterior  occipital  sites.  N258  probably 
reflects  the  same  processes  as  the  N250  elicited  by  Si,  perhaps  the  encoding  of  higher-order,  detailed 
physical  features  of  the  graphic  figures. 
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In  the  Si  epoch,  there  is  a  N568  peak,  largest  at  the  right  frontal  electrode  F4  (t(l,8)=4.18,  p<.05). 
S2  also  elicits  a  right-lateralized  N460  wave  largest  at  frontal  sites  F4  and  F6.  Thus  for  each  of  the 
two  stimuli  there  is  a  pattern  of  greater  left  posterior  activity  early  in  each  epoch,  possibly  reflecting 
the  encoding  of  complex,  higher-order  physical  characteristics  of  the  graphic  figures,  followed  by 
greater  right  anterior  activity  later  in  the  epoch,  possibly  related  to  the  maintainence  and  comparison 
of  the  graphic  codes,  and  to  response  selection. 

Activity  which  produced  larger  peaks  in  the  language  conditions  occured  first  in  the  P313  peak,  most 
distinct  at  left  frontal  sites  (F3  and  F5),  followed  by  the  N448  peaks  at  left  temporal  (T3),  anterior 
central  (aC5)  anterior  occipital  (aOl)  and  frontal  (F7)  electrodes  (Figure  7).  The  language  conditions 
did  not  differ  from  each  other  at  T3,  aC5  and  F7,  but  they  did  differ  among  themselves  at  aOl.  Thus 
the  significant  amplitude  difference  at  the  left  anterior  occipital  electrode  (aOl)  may  be  due  to  a  dif¬ 
ferent  process,  peaking  slightly  later  than  the  N448.  Since  the  language  conditions  do  not  differ  from 
each  other  at  T3,  aC5  and  F7,  we  might  speculate  that  the  N448  peaks  reflect  an  process  common  to 
all  language  conditions,  such  as  generation  of  phonemic  codes.  The  complex  of  activity  at  the  latency 
of  the  P279  peak  elicited  by  S2  contained  amplitude  increases  to  the  language  stimuli  at  left  frontal 
sites,  and  to  graphic  stimuli  at  midline  frontal,  right  anterior  sites,  and  posterior  electrodes  of  both 
right  and  left  hemispheres  (where  it  was  negative).  Since  this  peak  also  exhibited  significant  differ¬ 
ences  between  grammatic  and  semantic  conditions,  it  appears  that  the  P279  (and  its  negative  counter¬ 
part  over  posterior  regions)  is  sensitive  to  almost  every  condition  difference  in  this  experimental  para¬ 
digm.  Each  condition  causes  a  relative  amplitude  increase  at  one  or  more  electrodes  over  some  region 
of  cortex.  It  is  a  point  of  theoretical  interest  whether  this  is  the  result  of  several  different  processes 
occurring  co-incidently  at  the  same  latency,  or  whether  it  results  from  one  distributed  process  whose 
widely  separated  cortical  processing  nodes  respond  differently  to  the  various  task  requirements  of  the 
four  conditions.  This  is  more  than  a  moot  point,  and  in  fact  bears  on  the  whole  concept  of  distributed 
processing  networks  versus  notions  of  'parallel/sequentiaT  models. 

2.  Word  and  Non-word  Processing 

The  word/non- word  distinction  (semantic  and  grammatic  versus  graphic  and  phonemic  conditions) 
produced  differences  in  both  the  Si  and  S2  epochs  (Figure  8).  Significant  differences  occurred  at 
about  684  msec  after  Si  at  right  and  left  temporal  and  frontal  sites.  After  S2,  at  the  right  parietal, 
temporal  and  frontal  electrodes  (P4,  aP2,  F4,  T4,  aP6,  and  aF2)  at  about  365  msec,  and  at  left  lateral 
antero-central  (aC5),  left  temporal  (T3),  left  frontal  (F5),  and  anterior  occipital  (aOl)  electrodes  at 
about  563.  The  tendency  in  these  cases  was  for  the  graphic/phonemic  waveforms  to  have  larger 
amplitudes  than  the  semantic  and  grammatic.  It  is  possible  that  the  nonsense  phonemic  processing 
than  familiar  words  of  the  semantic  and  grammatic  conditions.  In  order  to  determine  whether  this 
effect  was  due  to  semantic  processing  or  increased  phonemic  processing  to  nonsense  words,  the  graphic 
and  phonemic  conditions  combined  were  compared  to  the  semantic  and  grammatic  conditions  com¬ 
bined,  and  the  graphic  and  phonemic  conditions  were  tested  for  differences.  It  was  iound  that  the 
graphic  and  phonemic  conditions  tended  to  behave  similarly  at  most  (but  not  all)  sites  where  there 
were  differences  between  the  word  and  ’nonsense  word’  (phonemic)  conditions.  Separation  of  the 
phonemic  non-word  condition  from  the  word  conditions  is  consistent  with  models  for  lexical  processing 
which  posit  activation  of  independent  phonological  and  semantic  pathways  for  visually  presented 
words  (Coltheart,  1985;  Petersen  et  al.,  1988).  Evidence  for  this  model  is  provided  by  dissociations  of 
letter  versus  word  reading  in  dyslexia  (Marshall,  1984),  and  by  access  to  phonological  features  of  lexi¬ 
cal  targets,  but  not  the  lexical  targets,  in  anomia  and  aphasia  (Farmer,  1978;  Illes  et  al.,  1986; 
Joanette  et  al.,  1980).  The  presence  of  bilateral  word/non-word  effects  is  not  surprising,  in  view  of  the 
abundant  evidence  for  basic  lexical  comprehension  in  the  non-dominant  hemisphere  (Gazzaniga  and 
Sperry,  1967;  Ardila  and  Ostrosky-Solis,  1984).  Somewhat  unexpected,  however,  was  that  the  effect 
occured  200  msec  earlier  over  the  non-dominant  hemisphere! 
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3.  Syntactic  Processing 

Grammatic  and  semantic  conditions  were  differentiated  after  the  first  stimulus  by  the  N442  peak 
Considering  that  both  types  of  stimuli  were  words  with  similar  physical  charactaristics,  the  main 
difference  was  that  the  semantic  condition  used  open  class  words  (content  words  such  as  nouns  and 
adjectives)  and  the  grammatic  condition  used  closed  class  words  (syntactic,  or  function  words)  in  this 
case  pronouns.  It  is  thus  reasonable  to  suppose  that  the  N442  is  related  to  processing  closed-class 
words,  to  the  initiation  of  a  'syntactic  parser"  (Garrett,  1982),  or  to  both.  The  "syntactic"  effect  at 
left  frontal  sites  (F3,  F5  and  aC3)  is  consistent  with  neurophysiological  observations  of  syntactic  defi¬ 
cits  and  difficulties  in  handling  closed  class  words  in  aphasia  patients  whose  lesions  involve  and  extend 
deep  to  Broca’s  area  (e.g.,  Gordon,  1985;  Metter  et  al.,  1983).  On  the  one  hand,  it  is  unlikely  that 
these  were  independent  processes  since,  based  on  cognitive  studies  with  healthy  and  aphasic  subjects, 
it  is  well  known  that  processing  of  closed  class  words  is  closely  linked  to  the  processing  of  grammatical 
form  (reviewed  e.g.  in  Lecours  et  al.,  1983).  One  the  other  hand,  the  existence  of  a  specialized  access 
system  responsible  for  retrieving  closed  class  items  during  language  processing  is  still  being  investi¬ 
gated  (e.g.,  Bradley  et  al.,  1980;  Gordon  and  Caramazza,  1983;  Petocz  and  Oliphant,  1988;  Matthei 
and  Kean,  1989).  The  highly  significant  effects  at  right  parietal  sites  is  puzzling,  and  appears  to  be 
part  of  a  different  process  peaking  at  a  later  latency. 

The  P279  which  distinguishes  the  grammatic  from  semantic  condition  after  the  second  stimulus  is  eli¬ 
cited  by  an  open  class  word,  specifically  a  verb  which  correctly  completes  the  meaningful  and  gram¬ 
matically  correct  sentences  of  the  match  trials.  One  possibility  is  that  P279  reflects  differences  in  pro¬ 
cessing  of  verbs  versus  stimulus  words  belonging  to  other  lexical  categories  (e.g.,  nouns,  adjectives, 
adverbs).  This  is  unlikely,  however,  since  65%  of  the  semantic  stimuli  were  verb  pairs  themselves,  or 
pairs  in  which  the  S2  could  be  either  a  verb  or  a  noun.  Thus,  the  alternative  and  most  likely  explana¬ 
tion  is  that  the  P279  may  reflect  syntactic  closure  of  processes  initiated  by  the  first  stimulus. 

4.  Comparison  of  Linked-Ears  EPs  and  Laplacian  Derivation  Topographies 

The  major  benefit  of  the  LD  is  that  it  acts  as  a  spatial  high-pass  filter,  reducing  common  activity  and 
enhancing  local  cortical  activity.  Thus  the  LD  averages  show  considerably  more  localized  effects  than 
the  linked-ears  EP  waveforms.  The  sharply  defined  effects  in  the  LD  waveforms  elicited  by  the  four 
conditions  in  this  study  stand  in  contrast  to  the  conflicting  reports  of  lateralizations  and  localizations 
in  previous  evoked-potential  language  studies  (reviews  in  Kutas  and  Hillyard,  1980b;  Regan,  1988). 
Although,  the  LD  distributions  should  not  always  be  literally  interpreted  as  reflecting  the  activity  of 
directly  underlying  cortical  areas,  their  correspondence  to  language  and  other  cortical  association  areas 
in  this  study  is  striking. 

5.  Conclusion 

By  designing  four  carefully  controlled  tasks,  recording  from  59  scalp  locations,  and  spatially  enhancing 
the  task-related  signals  using  the  Laplacian  Derivation,  an  unusually  detailed  picture  of  the  complex 
spatial  and  temporal  course  of  several  types  of  linguistic  processes  was  obtained.  The  language  condi¬ 
tions  exhibited  sharply  localized  activity  over  left  temporal  and  frontal  cortices.  All  three  language 
conditions  were  alike  in  that  they  elicited  similar  N448  peaks  at  the  left  temporal  electrode  (T3)  in 
response  to  the  first  stimulus.  However,  they  differed  among  themselves  in  the  amplitudes  of  other 
waves  which  were  maximal  at  left-frontal  (F3  and  F5)  and  anterior  central  (aC5),  and  temporal  (T3 
and  T4)  sites.  Although  the  linguistic  and  graphic  conditions  differed  greatly,  many  of  the  graphic 
effects  occured  at  left-sided  posterior  sites,  supporting  the  view  that  the  posterior  left-hemisphere 
mediates  the  processing  of  complex  visual  figures.  In  addition,  evidence  for  the  priming  of  visual  asso¬ 
ciation  cortex  in  anticipation  of  the  greater  processing  requirements  of  unfamiliar  graphic  figures  was 
found. 
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Figure  10.  Topography  of  the  Cue-evoked  P323  evoked  potential  peak  at  all  59  scalp  channels  [LEFT], 
and  corresponding  Laplacian  Derivation  N323  peak  at  the  39  non-peripheral  channels  [RIGHT].  The 
P323  EP  topography  differs  from  that  of  the  typical  oddball  'P300'  peak  in  its  anterior  negativity 
and  left-sided  lateralization  of  the  posterior  positivity.  The  site  of  maximum  amplitude  in  both  the 
potential  and  LD  distributions  is  at  the  left  anterior  occipital  electrode  (indicated  by  arrows).  The  LD 
distribution  also  contains  a  current  source  at  the  midline  frontal  electrode  (Fz).  [Scale:  EP,  -+-/-3.15 
microvolts;  LD,  -+-/-. 146  microvolts  per  square  centimeter.] 
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IV.  TECHNICAL  DEVELOPMENTS 


A.  Overview 

All  brain  signals  undergo  considerable  spatial  smearing  as  they  pass  through  the  tissues  between  the 
brain  and  scalp,  and  in  particular,  the  highly  resistive  skull.  The  popular  technique  of  equivalent 
dipole  localization,  which  exists  in  various  degrees  of  sophistication,  is  adequate  only  for  simple  sources 
such  as  early  exogenous  transient  responses  or  steady-state  potentials.  It  cannot  deal  with  multiple 
dipole-like  sources  or  complex  distributed  sources  without  a  great  deal  of  prior  knowledge  about  their 
location  and  orientation.  In  addition,  concentric  sphere  or  elipsoid  models  of  the  skull,  CSF  and  scalp 
do  not  resemble  the  actual  geometries  of  the  head  and,  moreover,  the  actual  thickness  of  these  tissues 
is  variable.  To  accurately  derive  a  picture  of  the  electrical  currents  of  the  brain  at  the  cortical  sur¬ 
face,  it  is  necessary  to  first  construct  a  more  accurate  3-D  model  of  each  subject’s  head  from  high 
quality  2-D  MRI  slices.  A  computationally  intensive  method  of  solving  the  Maxwell  equations  within 
a  large  number  of  3-D  polygons  ('finite  elements')  comprising  this  reconstruction  of  brain,  skull,  etc., 
must  then  be  applied.  This  method  of  'Finite  Element  Modeling"  requires  a  high  degree  of  spatial 
sampling,  at  least  on  the  order  of  120  or  more  electrodes.  This  laboratory  is  in  the  process  of  develop¬ 
ing  such  a  method  and  applying  it  to  simple  sensory  stimulation  as  the  first  step  toward  application  to 
complex  signals  generated  during  cognitive  functions.  Below  we  report  first  the  results  of  simple 
visual  dipole  localization,  and  then  our  progress  toward  a  more  advanced  deblurring  technology  in  a 
preprint  titled  'Beyond  Topographic  Mapping'. 


B.  Dipole  Localization 

It  is  well  known  from  anatomical  and  physiological  investigations  that  the  visual  field  (retina)  projects 
in  an  orderly  manner  onto  the  striate  area  of  the  occipital  cortex.  In  essence,  the  visual  field  is  pro¬ 
jected  contralaterally  and  inverted  about  the  horizontal  midline,  with  foveal  projections  located  at  the 
occipital  pole  and  the  more  peripheral  retina  projecting  to  cortex  located  deeper  within  the  calcarine 
and  longitudinal  fissures.  Evoked  potential  research  has  shown  that  the  early-latency  components  of 
the  visual  evoked  potential  (VEP)  for  pattern  stimuli  presented  to  restricted  retinal  areas  can  be 
accounted  for  by  equivalent  dipole  sources  located  in  corresponding  retinotopic  projection  areas. 
Evoked  magnetic  field  data  have  indicated  similar  localization,  primarily  based  on  steady-state  stimu¬ 
lation  and  analysis  methods.  This  study  evaluated  dipoie  localization  for  the  steady-state  evoked 
potential  to  pattern  reversal  stimulation  of  all  visual  quadrants  and  octants  in  4  subjects. 

Quadrant  and  octant  patterns  (Figure  11,  TOP)  consisted  of  pie-shaped  sections  of  a  circular  black- 
and-white  checkerboard  having  8  concentric  tracks  each  extending  2-degrees  from  a  central  fixation 
point.  Patterns  reversed  in  phase  15  times  per  second  on  a  computer  monitor  1  meter  in  front  of  the 
subject.  Duration  of  stimulation  was  either  50  or  100  seconds,  and  three  stimulation  runs  were 
obtained  for  each  pattern.  Data  recorded  from  the  4  normal  male  subjects  consisted  of  horizontal  and 
vertical  eye-movements  (EOG)  as  well  as  24  channels  of  EEG  (referenced  to  Fz)  selected  from  an 
extended  10/20  system,  including  17  electrodes  on  the  posterior  scalp.  Artifact-free  epochs  of  0.533 
sec,  digitized  at  128/s,  were  averaged  and  Fourier-transformed.  Real  and  imaginary  components  of 
the  15-Hz  response  were  drawn  onto  maps  of  the  head  for  the  3  averages  from  each  subject  and 
stimulus,  and  for  the  grand  average  of  the  three  stimulation  runs.  Data  for  each  pair  of  adjacent 
octants  and  the  corresponding  quadrant  were  then  combined  for  calculation  of  the  'canonical”  phase 
(i.e.,  the  phase  angle  onto  which  the  projections  of  all  the  phasors  yielded  the  greatest  squared  ampli¬ 
tude).  For  dipole-fitting,  the  theoretical  position  of  each  electrode  on  a  sphere  centered  halfway 
between  Fpz  and  Oz  was  calculated.  Least-squares  analysis  was  used  to  find  the  location  of  the  best¬ 
fitting  current  dipole  within  the  sphere  at  a  resolution  of  2.5  mm. 
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The  steady-state  amplitude  measures  from  the  phasor  analysis  were  used  to  construct  topographical 
isopotential  maps  and  for  least-squares  fitting  of  equivalent  current  dipoles.  Figure  11  [BOTTOM] 
shows  the  electrode  locations  in  a  rear  view  of  the  head.  Figure  12  [TOP]  presents  maps  in  this  view 
for  each  of  the  stimuli  for  one  subject.  Also  shown  (Figure  12,  BOTTOM)  is  the  location  in  the 
coronal  plane  of  the  computed  dipoles,  and  also  its  direction  for  that  subject.  The  length  of  the  3-D 
extent  of  each  dipole  was  set  constant;  the  variation  in  length  is  due  to  the  direction  angle  out  of  the 
coronal  plane.  Figure  13  summarizes  of  these  dipole  fittings  by  averaging  the  computed  dipole  param¬ 
eters  for  the  four  subjects. 

These  results  show  potential  topographies  consistent  with  occipital  generators,  and  very  close  linear 
summation  of  adjacent  octant  responses  compared  to  those  for  corresponding  quadrants.  Each 
equivalent  dipole  fit  for  each  subject  (with  the  exception  of  just  one  octant  combination)  accounted 
for  81%  of  the  variance  in  the  response.  These  dipole  locations  were  near  the  occipital  pole  contrala¬ 
teral  to  the  side  of  stimulation  and  also  showed  inversion  with  respect  to  the  horizontal  meridian. 
These  results  are  comparable  to  those  found  for  early-latency  transient  EPs  and  for  evoked  magnetic 
sources.  However,  the  EEG  equipment  for  this  type  of  dipole  analysis  costs  about  100  times  less  than 
MEG  equipment  with  a  comparable  number  of  channels.  There  were  savings  also  in  terms  of  time, 
since  only  about  5  minutes  of  data  was  required. 

Although  this  method  proved  very  efficient  and  effective  in  locating  dipoles  for  simple  steady-state 
sensory  stimuli,  single  equivalent  dipoles  are  not  a  suitable  model  for  the  complex,  distributed 
processes  of  cognitive  functions.  Work  is  already  underway  in  this  labors/ "tv  m  the  development  of 
more  realistic  models,  as  described  below. 


/«-g;/rcports/afosr/final.90 


-35- 


FINAL  TECHNICAL  REPORT 


APR87-MAR90 


EEG SYSTEMS  LABORATORY 


F49620-87-C-0047 


Figure  11.  [TOP]  -  Visual  stimulus  patterns;  [BOTTOM]  -  Electrode  montage  viewed  from  the  back  of 
the  head. 
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Figure  12.  Potential  field  topographies  corresponding  to  the  stimuli  in  Figure  12,  from  one  subject 
[TOP],  and  the  locations  in  the  coronal  plane  and  directions  of  the  corresponding  computed  dipoles  for 
that  subject  (BOTTOM|. 
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Figure  IS.  Dipole  locations  for  the  8  octant  stimuli  averaged  across  the  four  subjects  are  shown  as 
colored  cylinders.  The  rear  surface  of  the  head  has  been  removed  to  view  the  dipole  positions  and  orien¬ 
tations  within.  Direction  of  each  dipole  is  indicated  by  the  central  axis  of  each  flat  cylinder.  The  large 
multicolor  circle  is  a  color  legend  indicating  the  visual  stimulus  octant  corresponding  to  each  dipole.  In 
order  to  make  the  retinotopic  projection  easier  to  visualize,  the  color  legend  depicts  which  sectors  in  each 
retina  were  stimulated  (e.g.,  blue  colors  indicate  upper-right  retinal  stimulation  resulting  from  a  check¬ 
erboard  in  the  lower-left  visual  field.) 
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C.  Beyond  Topographic  Mapping:  Towards  Functional- Anatomical  Imaging 
with  124-Channel  EEG’s  and  3-D  MRJ’s 
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ABSTRACT 

A  functional-anatomical  brain  scanner  that  has  a  temporal  resolution  of  less  than  a  hundred 
milliseconds  is  needed  to  measure  the  neural  substrate  of  higher  cognitive  functions  in  healthy  people 
and  neurological  and  psychiatric  patients.  Electrophysiological  techniques  have  the  requisite  temporal 
resolution  but  their  potential  spatial  resolution  has  not  been  realized.  Here  we  briefly  review  progress 
in  increasing  the  spatial  detail  of  scalp-recorded  EEGs  and  in  registering  this  functional  information 
with  anatomical  models  of  a  person’s  brain.  We  describe  methods  and  systems  for  124-channel  EEGs 
and  magnetic  resonance  image  (MRI)  modeling,  and  present  first  results  of  the  integration  of 
equivalent-dipole  EEG  models  of  somatosensory  stimulation  with  3-D  MRI  brain  models. 
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INTRODUCTION 

There  are  a  number  of  techniques  for  monitoring  brain  function  that  are  more  or  less  noninvasive, 
including  Positron  Emission  Tomography  (PET),  Single  Photon  Emission  Computed  Tomography 
(SPECT),  Magnetic  Resonance  Spectroscopy  (MRS),  Electroencephalography  (EEG)  and 
Magnetoencephalography  (MEG).  Although  the  3-Dimensional  (3-D)  anatomical  imaging  capabilities 
of  EEG  or  MEG  are  not  comparable  to  PET,  SPECT  or  MRs,  EEGs  and  MEGs  uniquely  offer 
temporal  resolution  in  the  millisecond  range.  This  particular  capability  is  invaluable  for  studying 
seizure  disorders  and  neurocognitive  processes.  With  improving  recording  and  analysis  technologies, 
other  valuable  information  that  generations  of  clinicians  and  researchers  over  the  past  60  years  have 
believed  was  hidden  in  the  EEG  is  becoming  more  accessible  (see  reviews  in  Lopes  da  Silva  et  al.,  1986; 
Gevins  and  Remond  1987).  Indeed,  as  technical  capabilities  have  increased,  so  has  the  specificity  of 
information  extracted  from  EEGs,  and  the  value  of  further  developing  methods  to  mine  this  hidden 
information  is  clearly  mandated  by  the  noninvasiveness  and  low  cost  of  scalp  EEG  measurements  for 
obtaining  split-second  information  about  brain  function. 

While  spatial  EEG  features  have  often  been  difficult  to  discern  in  polygraph  tracings,  the  recent 
availability  of  EEG  color  topographic  maps  has  made  appreciating  them  easy,  even  for  the  non-EEG 
specialist.  The  utility  of  the  maps  would  be  even  greater  if  more  spatial  detail  were  available,  and  if 
the  scalp  EEG  patterns  could  be  registered  with  underlying  brain  anatomy.  During  the  past  8  years, 
we  have  made  some  modest  progress  towards  this  end.  We  have  improved  and  expanded  our 
recording  and  analysis  capabilities  to  accommodate  124  EEG  channels,  and  have  developed  methods 
for  reducing  blur  distortion  of  EEGs  by  the  skull.  We  have  implemented  programs  for  processing 
magnetic  resonance  images,  and  we  are  developing  a  means  of  automatically  constructing  three- 
dimensional  (3-D)  models  of  a  subject’s  brain.  In  tests  of  our  metnod  for  registering  EEGs  and  MRIs 
using  somatosensory  stimulation  data,  we  obtained  good  results.  Previous  reports  of  prior  stages  of 
progress  in  this  regard  have  been  published  in  Gevins  (1987,  1988,  1989a, b). 


EEG  RECORDING  METHOD 

A.  Extended  10-20  System 

Several  nomenclatures  for  defining  electrode  placements  added  to  the  original  nineteen  placements  of 
the  International  10-20  System  (Jasper,  1958)  have  been  published  (see  Gevins,  1988).  Regardless  of 
which  nomenclature  is  used,  the  basic  idea  is  that  extra  equidistant  coronal  rows  are  added  between 
the  original  rows,  and  extra  equidistant  electrodes  are  added  to  fill  in  the  spaces  in  each  row.  Our 
current  124-channel  montage  is  shown  in  Figure  1.  In  the  system  we  use,  the  letter  "a"  is  prefixed  to 
an  existing  10-20  row  name  to  indicate  a  position  anterior  to  the  existing  one.  For  example,  aO  is 
anterior  occipital,  aP  is  anterior  parietal,  aC  is  anterior  central  and  aF  is  anterior  frontal.  Two 
additional  numbers  appear  after  the  location  number  (1-8)  of  the  10-20  positions.  These  two  numbers 
(0-9)  indicate  the  proportional  distance  to  the  next  anterior  and  medial  10%  electrode  position, 
respectively.  For  example,  an  electrode  halfway  between  P3  and  aPl  would  be  named  P355,  while  an 
electrode  halfway  between  P3  and  Pi  would  be  named  P305.  With  the  original  nineteen  electrodes  of 
the  10-20  System,  the  typical  distance  between  electrodes  on  an  average  adult  male  head  is  about  6 
cm;  with  124  electrodes,  the  typical  distance  is  2.25  cm. 

B.  Electrode  Hat 

In  most  routine  clinical  EEGs,  electrodes  are  prepared  and  positioned  on  the  head  individually,  while 
in  some  labs  commercially  available  electrode  caps  with  nineteen  built-in  electrodes  are  used. 
Extending  the  cap  idea,  we  constructed  one  from  a  stretchable  fabric,  and  populated  it  with  124 
commercial  tin-disk  electrodes  encased  in  plastic  holders  (Figure  2).  The  cap  is  positioned  on  the  head 
by  reference  to  the  nasion,  inion  and  preauricular  notches.  Individual  recording  sites  are  cleansed  in 
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the  usual  manner,  and  conducting  gel  is  injected  with  a  blunt  needle  through  the  top  of  each  electrode. 
It  takes  a  team  of  four  research  assistants  about  an  hour  to  prepare  a  subject.  While  this  is  acceptable 
for  infrequent  research  recordings,  it  is  far  too  long  and  costly  for  routine  use.  We  are  developing  a 
more  efficient  system  for  recording  EEGs  which  we  expect  to  greatly  reduce  application  time. 

C.  Measuring  Electrode  Positions 

Traditionally,  individual  electrodes  are  placed  according  to  measurements  taken  with  a  tape  measure. 
While  this  is  sufficient  when  recording  from  19  electrodes,  greater  precision  is  needed  when  many  more 
electrodes  are  used  and  when  one  wishes  to  relate  a  recording  position  on  the  scalp  to  the  underlying 
cortical  anatomy.  In  our  lab,  the  position  of  each  electrode  on  a  subject’s  head  is  measured  with  a 
probe  that  has  coils  for  sensing  the  three-dimensional  position  of  the  probe  tip  with  respect  to  a 
magnetic  field  source  in  the  head  support  (Figure  3).  Adjustable  guides  built  into  the  head  support 
hold  the  subject’s  head  comfortably  in  place  while  the  measurements  are  made.  A  menu-driven 
program  is  used  to  select  electrodes  to  be  measured  and  display  the  digitized  position  of  each  electrode 
on  a  two-dimensional  projection  display.  Position  measurement  is  accurate  to  better  than  3mm  [RMS]. 
Figure  4  shows  the  screen  image  of  this  program  after  all  electrodes  and  some  additional  surface 
landmarks  have  been  digitized. 


D.  Software  System  for  Data  Collection  and  Analysis 

Commercial  EEG  computer  systems  have  vastly  improved  during  the  past  few  years,  yet  some 
important  limitations  stiil  restrict  their  utility  for  researchers  such  as  ourselves.  This  is  a  consequence 
of  the  fact  that  most  systems  have  been  designed  for  the  clinical  market  where  the  central  concern  is 
to  provide  a  series  of  fixed  tests  and  measures.  The  most  severe  limitations  are  that:  (l)  a  way  to 
implement  new  experimental  protocols  flexibly  is  rarely  provided;  (2)  artifact  detection  and  editing 
capabilities  are  too  limited;  (3)  the  system  capacity  for  recording  large  numbers  of  channels  and 
collecting  and  storing  large  databases  is  inadequate;  (4)  there  are  limited  means  of  subdividing  data  to 
explore  relationships  between  subsets  of  data  and  variations  in  a  subject’s  responses  or  state;  and  (5) 
spatial  analyses  are  underdeveloped,  with  too  few  channels,  a  lack  of  spatial  filters  to  reduce  volume 
conduction  distortion,  a  lack  of  cross-channel  analyses  (e.g.,  crosscovariance  and  crosspower,  correlation 
and  coherence),  and  the  lack  of  means  to  investigate  the  relationship  between  neuroelectric  data  and 
cortical  anatomy  and  physiology  revealed  by  MRI  and  PET  imaging  technologies. 

The  system  we  have  been  developing,  now  in  its  fifth  generation,  is  aimed  at  overcoming  these 
limitations  (Gevins  &  Yeager,  1972;  Gevins  et  ah,  1975,  1989;  Gevins,  1980,  1984,  1987;  1988)  (Figure 
5).  This  generation  is  distinguished  from  its  predecessors  architecturally  by  its  self-decoding  Data 
Description  Language,  network  extensibility,  and  multi-window  graphical  user  interface.  As  of  May 
1990,  all  the  functions  described  in  the  following  two  paragraphs  are  fully  operational. 


1.  Data  Collection:  In  the  fifth  generation  EEGSL  system,  two  computers  are  used  for  data 
collection:  a  Concurrent  5700  and  an  IBM  PC-compatible  386.  The  PC  is  used  to  present  stimuli  and 
gather  behavioral  response  data  from  the  subject,  while  the  Concurrent  collects  physiological  data  and 
controls  the  PC.  The  Concurrent  runs  a  real-time  version  of  the  UNIX  operating  system,  while  the  PC 
runs  the  DOS  operating  system.  We  have  written  an  experiment  control  software  system  which  runs 
the  two  computers,  presents  a  variety  of  visual,  auditory  and  somatosensory  stimuli  according  to 
flexible  task  protocols,  and  digitizes  up  to  256  channels  of  physiological  data.  Up  to  128  EEG  traces 
can  be  monitored  in  real  time  as  data  are  collected.  Most  parameters  of  an  experiment  can  be  altered 
via  a  menu-driven  interactive  display.  A  calibration  module  numerically  adjusts  the  gain  of  all 
channels  according  to  the  magnitude  of  a  calibration  signal.  Another  module  detects  gross  artifacts 
and  color-codes  contaminated  data  such  as  eye  movements,  gross  head  and  body  movements  and  bad 
electrode  contacts  on  the  operator’s  screen.  Stimulus,  behavioral  and  physiological  data  are  stored  on 
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hard  disk  according  to  a  self-decoding  Data  Description  Language,  and  archived  to  magnetic  tape. 
The  data  are  immediately  available  to  researchers  at  their  desks  using  either  remote  terminals,  or  SUN 
workstations  via  Ethernet  and  Network  File  System. 

2.  Data  Analysis:  For  subsequent  data  analysis  the  current  system  also  has  a  number  of  other 
functional  improvements  over  its  predecessors  including:  (1)  least-squares  Laplacian  derivation 
estimation  to  reduce  volume  conduction  distortion  (Gevins,  1989b);  (2)  digital  filters  with  user-specified 
characteristics;  (3)  time  series  analysis  including  spectral  analysis,  Wigner  Distributions  (Morgan  and 
Gevins,  1986),  and  event-related  covariance  analysis  (Gevins  et  al.,  1987,  1989a;  Gevins  and  Bressler, 
1988);  (4)  neural-network-driven  pattern  recognition  to  extract  optimal  or  near-optimal  subsets  of 
features  for  recognizing  different  experimental  or  clinical  categories  (Gevins,  1980,  1987;  Gevins  and 
Morgan,  1988);  and  (5)  anatomical  modeling  to  construct  3-D  finite  element  models  of  the  brain  and 
head  from  MRI  scans  (Le  et  al.,  In  Prep.;  Brickett  et  al.,  In  Prep.;  Reutter  et  al.,  In  Prep.).  Four  on¬ 
line,  interactive  subsystems  are  used  to  examine  and  edit  data  for  residual  artifacts  (on  an  individual 
channel  basis  if  desired),  sort  data  according  to  stimulus,  response  or  other  categories,  perform 
exploratory  data  analysis,  and  produce  three-dimensional  color  graphics  representations  of  the  brain 
and  head. 


MRI  ANALYSIS  AND  MODELING  METHODS 

Since  commercial  MRI  analysis  packages  are  not  designed  for  research  on  functional-anatomical 
integration  and  thus  lack  features  essential  to  our  undertaking,  we  have  had  to  develop  our  own 
algorithms  and  software  to  produce  3-D  brain  models  suitable  for  functional  localization  studies. 
Visualization  software  permits  construction  of  3-D  composites  of  multiple  2-D  image  planes,  as  well  as 
3-D  surface  rendering  based  on  surface  contours.  Since  generating  surface  contours  manually  is 
laborious  and  subject  to  error,  we  have  worked  on  automating  the  procedure.  We  have  also  automated 
the  alignment  of  the  digitized  EEG  electrode  positions  with  the  scalp  surface  contours,  which  is  a 
critical  first  step  in  a  functional-anatomical  analysis.  The  MRI  contour  information  is  used  to  produce 
a  mathematical  finite  element  model  (FEM)  of  the  brain  and  head  suitable  for  equivalent  dipole  source 
localization  and  scalp  EEG  deblurring  development.  Since  complex  FEM  normally  run  on 
supercomputers,  we  made  a  significant  effort  to  develop  FEM  algorithms  and  programs  that  could  run 
efficiently  on  a  desktop  workstation. 


A.  EEG-MRI  alignment  procedure 

In  order  to  visualize  the  brain  areas  underlying  EEG  electrodes,  a  procedure  is  needed  for  aligning 
scalp  electrode  positions  with  the  MR  Images.  In  the  procedure  we  now  use,  x,  y,  z  translation  and  x, 
y,  z  axis  rotations  are  computed  iteratively  to  align  the  digitized  positions  of  the  EEG  electrodes  with 
the  MRI  data.  This  is  done  for  each  electrode  by  finding  the  distance  to  the  closest  point  on  the  scalp 
surface  MRI  contours  and  minimizing  the  mean  distance  for  all  electrodes.  With  MR  images  that  have 
a  3  mm  inter-slice  spacing,  we  usually  achieve  a  mean  error  distance  better  than  2  mm.  This  is  more 
accurate  and  less  subjective  than  alignment  procedures  that  use  skull  landmarks  such  as  the  nasion 
and  pre-auricular  points  located  visually  in  the  MR  images.  Figure  6  shows  the  electrodes  displayed 
schematically  on  a  scalp  surface  reconstructed  from  horizontal  MR  scalp  contours  as  described  below. 

B.  3-D  Composite  MRI  displays 

Figure  7  shows  examples  of  3-D  composite  images  taken  from  a  routine  clinical  MR  exam  of  a  patient 
with  partial  seizures.  A  combination  of  horizontal,  coronal,  and  sagittal  plane  images  are  shown  in  five 
different  views.  Figure  8  shows  another  composite  from  the  same  data  and  also  a  partial  surface 
reconstruction  done  as  described  below. 
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C.  Contour  extraction  and  surface  rendering 

Both  the  scalp  and  cortical  surface  contours  in  Figures  7  and  8  were  traced  using  an  intensity 
thresholding  technique,  one  of  the  two  image  analysis  methods  we  developed  to  extract  scalp  and 
cortical  surface  contours  from  MRIs.  The  first  technique  uses  intensity  thresholding,  which  involves 
extracting  contours  along  which  the  image  intensity  is  equal  to  a  defined  threshold.  This  technique  is 
useful  for  extracting  the  scalp  surface  contour  which  can  be  discerned  easily  from  the  black 
(approximately  zero  intensity)  background  of  the  image. 

The  current  method  of  surface  rendering  has  six  steps:  1)  for  each  pair  of  adjacent  contours,  find  the 
point  on  the  second  contour  closest  to  the  first;  2)  calculate  the  two  distances  from  each  of  these 
points  to  the  next  point  on  the  adjacent  contour;  3)  make  a  triangle  using  the  point  with  the  shortest 
distance,  advancing  on  that  contour;  4)  repeat  steps  two  and  three  until  all  points  are  exhausted;  5) 
repeat  steps  two  to  four  in  the  reverse  direction  along  the  contours;  and  6)  piece  together  the  "best" 
result.  This  was  the  surface  reconstruction  method  used  for  Figure  8  showing  a  surface  reconstruction 
of  the  cerebral  cortex  (in  yellow),  from  the  same  patient  as  in  Figure  7,  using  the  available  coronal 
slices. 

The  second  contour  extraction  method  we  have  used  involves  differential  intensity  analysis.  Using  this 
technique,  contours  that  separate  image  regions  with  different  local  average  intensities  are  extracted. 
Resulting  contours  pass  through  the  points  in  the  image  at  which  the  local  average  image  intensity  is 
changing  most  rapidly.  This  technique  requires  no  a  priori  intensity  threshold  value,  and  is  useful  for 
extracting  the  cortical  surface  contour  which  has  a  less  well-defined  image  intensity  value  throughout 
an  image  than  the  scalp  surface  contour.  The  first-order  and  second-order  partial  derivatives  of  the 
image  are  estimated  using  2-D  filters,  and  these  derivatives  are  used  to  locate  the  local  maxima  in  the 
gradient  of  the  image  intensity.  Highly  computationally  efficient  filtering  techniques  have  been 
developed  for  the  estimation  of  the  partial  derivatives  of  the  image  (Algazi  et  al.,  1989).  Figure  9 
shows  an  example  of  contours  corresponding  to  locations  of  local  maxima  in  the  gradient  of  the  image 
intensity,  which  trace  sulci. 


D.  3-D  Cortical  surface  image  model 

3-D  surface  models  of  the  external  convexity  of  the  cerebral  cortex,  such  as  those  of  Figure  8,  are  not 
sufficient  to  visualize  and  computationally  model  the  cortical  surface  within  fissures  and  sulci.  As  a 
result,  we  developed  an  algorithm  to  model  cubic  volume  elements  (voxels).  The  faces  of  the  voxels  lie 
in  the  horizontal,  coronal,  and  sagittal  planes  for  which  MRI  data  have  been  obtained.  The  MR1  data 
are  mapped  onto  the  faces  of  the  voxels  to  obtain  a  3-dimensional  display  of  the  image  data.  The 
image  planes  used  are  averaged  horizontal  planes  that  lie  halfway  between  the  horizontal  planes  in 
which  MRI  data  have  been  acquired,  and  coronal  and  sagittal  image  planes  that  are  synthesized  from 
the  acquired  horizontal  images  using  linear  interpolation.  The  spacing  between  image  planes  is  3  mm, 
which  yields  voxels  with  dimensions  of  3  mm  by  3  mm  by  3  mm.  The  images  have  pixel  dimensions  of 
approximately  1  mm  by  1  mm. 

The  initial  set  of  voxels  is  the  set  bounded  by  those  voxels  that  lie  just  inside  the  cortical  surface.  To 
view  the  MRI  data  at  a  slightly  deeper  level,  a  mathematical  morphology  erosion  operation  is  used  to 
remove  the  boundary  layer  of  voxels,  thereby  exposing  the  faces  of  the  voxels  that  lie  one  layer  deeper. 
By  eroding  the  model  iteratively,  anatomical  structures  can  be  tracked  and  3-dimensional  models  of 
the  structures  can  be  made  (see  Figure  12). 


eeK/pf°po«Ala/oiir/topog.oiir 


-6- 


Gevins  et  aj.:  Beyond  Topographic  Mapping 


MAY90 


E.  Finite  Element  Modeling 

Maxwell’s  Equation,  (Landau  et  al.,  1984,  Nunez,  1981) 

V  •  (  crVu  )  +  V  •  J  =  0  in  ft  (l) 

is  frequently  used  to  study  the  electromagnetic  field  generated  by  populations  of  neurons.  Here  a  (> 
0)  is  the  conductivity  tensor  at  a  point  R  —  (x,y,z)  in  H  (e.g.,  a  human  head),  u  is  the  electric  poten¬ 
tial  at  R,  and  J  is  the  electric  current  density  at  R.  When  f2  represents  a  human  head,  the  following 
boundary  condition  for  equation  (1)  is  obtained 

crVu*n  =  0  on  S  (2) 


since  electric  current  does  not  flow  out  of  the  head  in  the  direction  normal  to  the  surface.  The  limited 
scalp  potential  measurement  specifies  another  boundary  condition. 

u  =  U(  x,  y,  z  )  on  S{.  (3) 

Where  is  a  subsurface  of  S  corresponding  to  area  covered  by  electrodes. 

The  Finite  Element  Method  (FEM)  we  developed  finds  the  numerical  solution  of  equation  (l)  with  the 
boundary  condition  as  specified  in  equation  (2)  when  the  electromagnetic  field  activities  of  a  head  are 
modeled.  Advantages  of  this  approach  are:  1)  it  handles  the  Dirac  Delta  Function  smoothly  by 
transforming  equation  (1)  into  a  variational  form  when  J  represents  a  dipole-like  kind  of  source;  2)  it 
allows  us  to  model  the  complicated  geometries  of  different  tissues  within  the  head  by  generating  finite 
elements  using  contours  on  pairs  of  adjacent  MR  images;  and  3)  it  produces  a  sparse  matrix  in  which 
many  entries  are  zeroes.  We  developed  efficient  sparse  matrix  techniques  (Le  et  al.,  in  prep.)  that  can 
solve  Maxwell’s  equation  at  10,087  FEM  nodes  in  about  100  min  on  a  SUN  Sparc-1  workstation  (12 
MIPS)  with  8MB  of  memory  (Figure  10). 

We  can  use  this  analysis  to  localize  single  equivalent  dipole  sources  of  recorded  scalp  potentials,  or 
obtain  the  potential  distribution  within  the  whole  head  given  the  localized  source.  To  do  this,  we 
choose  a  specific  function  J  and  solve  the  corresponding  potential  distribution  u  using  FEM  such  that 
the  difference  between  u  on  and  U(x,y,z)  defined  on  is  minimized  using  an  iterative  process. 
Another  use  of  FEM,  in  which  the  current  distribution  on  the  surface  of  the  brain  is  estimated  from 
scalp  recordings,  is  described  in  the  next  section. 


TOWARDS  FUNCTIONAL-ANATOMICAL  INTEGRATION 
A.  Spatial  filters  to  reduce  blur  distortion 

Electrical  currents  generated  by  sources  in  the  brain  are  volume  conducted  through  brain,  cerebrospi¬ 
nal  fluid,  skull  and  scalp  to  the  recording  electrodes.  Because  of  this,  potentials  due  to  a  localized 
source  are  spread  over  a  considerable  area  of  scalp  and  the  potential  measured  at  a  scalp  site  represents 
the  summation  of  signals  from  many  sources  over  much  of  the  brain.  Using  a  4-sheli  spherical  head 
model,  we  have  estimated  the  "point  spread'  for  a  radial  dipole  in  the  cortex  to  be  about  2.5  cm 
(Doyle  and  Gevins,  Submitted).  The  "simplest"  way  to  reduce  this  distortion  is  to  take  the  Laplacian 
in  two  dimensions  about  each  electrode.  Details  of  this  method,  however,  are  less  straightforward  than 
they  might  seem,  as  discussed  below.  Another  improvement  in  distortion  is  possible,  in  principle,  by 
using  a  Finite  Element  Method  and  estimates  of  the  conductivity  properties  of  the  tissue  between 
brain  and  scalp  surface  to  estimate  the  potentials  which  would  actually  be  recorded  on  the  surface  of 
the  brain. 
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1.  Laplacian  operator:  Neuroelectric  signals  recorded  at  the  scalp  are  principally  distorted  by 
transmission  through  the  low-conductance  skull.  This  distortion  manifests  as  a  spatial  low-pass  filtering 
which  causes  the  potential  distribution  at  the  scalp  to  appear  blurred  or  out  of  focus.  There  are  a 
number  of  methods  for  reducing  this  distortion,  among  which  the  spatial  Laplacian  operator  is  perhaps 
the  simplest  and  most  effective  (Nunez,  1981).  This  method,  which  is  often  referred  to  as  the  Lapla¬ 
cian  Derivation,  is  derived  by  computing  the  second  derivative  in  space  of  the  potential  field  at  each 
electrode.  This  converts  the  potential  into  a  quantity  proportional  to  the  current  entering  and  exiting 
the  scalp  at  each  electrode  site,  and  eliminates  the  effect  of  the  reference  electrode  used  during  record¬ 
ing.  An  approximation  to  the  Laplacian  Derivation,  introduced  by  Hjorth  (1975,  1980)  assumes  that 
electrodes  are  equidistant  and  at  right  angles  to  each  other.  Although  this  approximation  is  fairly  good 
for  some  electrode  positions  such  as  midline  central  (Cz),  it  is  less  accurate  for  others  such  as  midtem¬ 
poral  (T5).  We  have  been  using  a  more  accurate  estimate  of  the  Laplacian  that  is  based  on  projecting 
the  measured  electrode  positions  onto  a  two-dimensional  surface.  Although  this  produces  a  dramatic 
improvement  in  topographic  detail,  some  problems  remain  because  of  the  assumptions  that  surround¬ 
ing  electrodes  used  to  estimate  the  Laplacian  of  an  electrode  are  near  that  electrode  and  that  the 
current  gradient  is  uniform  over  the  region  encompassed  by  the  surrounding  electrodes.  Furthermore,  it 
is  not  possible  to  estimate  the  Laplacian  at  peripheral  electrodes  since  there  are  no  surrounding  elec¬ 
trodes  on  one  side. 


The  objective  of  the  LD  calculation  is  an  accurate  estimate  of 


dx 2  dy2 


(1) 


When  the  electrodes  are  arranged  in  a  uniform  square  grid,  the  method  described  by  Hjorth  will  suf¬ 
fice.  However,  both  variations  in  distance  and  variations  in  angle  are  always  present  to  some  degree, 
and  can  cause  serious  errors  in  calculation.  Variations  in  distance  can  be  corrected  by  dividing  by  the 
interelectrode  distance  (Thickbroom,  1984).  Non-uniform  angles  between  electrodes  are  more  difficult 
to  correct.  Spline  extrapolation  methods  have  been  used  to  account  for  both  types  of  variation  (Per¬ 
rin,  1987),  but  require  such  large  amounts  of  computation  that  they  are  practical  only  for  small 
amounts  of  data.  The  method  presented  below  also  accounts  for  both  types  of  variation,  but  requires 
only  an  initial  large  computation  of  coefficients  based  on  the  actual  geometry  of  the  electrodes. 

Assume  we  wish  to  compute  the  LD  at  position  q0  using  N  surrounding  electrodes  in  an  arbitrary  con¬ 
figuration  as  shown: 

Let  the  voltage  at  position  q be  designated  by  V).  Put  the  origin  of  a  cartesian  coordinate  system  at 
<7o  with  the  z  axis  pointing  perpendicular  to  the  plane  nearest  the  electrodes.  Let  (?u,?,y)  represent  the 
x  and  y  components  of  the  electrode  position  within  this  plane. 

Then  E(x,y)=  -VV(z,y)  is  the  electric  field  in  the  plane  or  equivalently: 


V0-Vi  =  J  E- ds 
c 


(2) 


Where  C  is _any  curve  in  the  plane  connecting  qo  to  If  we  let  Ez  and  Ey  designate  the  x  and  y  com¬ 
ponents  of  E  then  the  two-dimensional  laplacian  of  the  voltage  is  given  by: 


V2V  =  - 


dEz  dEy 
dx  +  dy 


(3) 
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The  field  E  can  be  expanded  in  a  two-dimensional  Taylor  series  (Rudin,  1976;  Apostol,  1969)  as: 

fro  =  m  +  Dm-,  +  ii?[i2?m  (4) 

where  qT  —  [qz,qy)  ,~r*is  a  remainder  term  which  is  bounded  as  q— *0  and  DE  is  the  Jacobian  matrix  of 
E  given  by: 

dEx  dEz 
dx  dy 

dEy  dEy 
dx  dy 

If  we  let  a(t)  be  a  parameterization  of  the  line  segment  connecting  <?,-  with  q0  then  taking  the  first  two 
terms  in  equation: 


We  can  combine  the  above  equation  for  each  of  the  surrounding  electrodes  in  a  single  matrix  equation: 


*7lz  Qly 

?ix2/2 

9lx9ly 

9ly2/2  1 

£*(0) 

'Vo-Vi 

?2z  ?2y 

72x2/2 

?2x?2y 

?2y2/2 

Ey(o) 

v0-v2 

dEx 

dx 

9ny2/2 

dEz 

Qnz  Qny 

<7nx2/2 

7nx  Qny 

dy 

v0-vn 

dEy_ 
■  dy 

QF  =  V 


(8) 


(9) 
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In  this  equation  the  Q  matrix  is  composed  of  the  x  and  y  coordinates  electrode  positions  and  remains 
fixed  during  the  entire  recording.  The  voltage  vector  V  is  measured  at  each  time  point  and  vector  F 
composed  of  the  electric  field  components  and  their  derivatives  is  unknown. 


Multiplying  the  above  by  the  transpose  of  Q  gives 

QtQF=QtV  (10) 

If  Q  premultiplied  by  its  transpose  QT  is  nonsingular  then  we  can  form  the  least  squares  estimate  of  F 
as 


F  = 


(11) 


If  we  let  C  be  the  sum  of  the  third  and  fifth  rows  of 
can  estimate  the  laplacian  LD  as: 


Q  T  then  from  the  above  equation  we 


V2  V 


dEz  dEy 
dx  dy 


-CV 


(12) 


2.  Finite  Element  Method  deblurring  in  a  realistic  head  model:  Since  an  "image"  of  the  potential 
distribution  over  local  cortical  areas  is  highly  desirable,  spatial  deblurring  methods  can  be  used  to 
reduce  volume  conduction  and  increase  spatial  resolution.  One  method  of  doing  this  without  introduc¬ 
ing  an  arbitrary  model  of  the  actual  number  and  location  of  sources  is  to  use  finite  element  methods  to 
represent  the  true  geometry  of  cortex,  cerebrospinal  fluid,  skull  and  scalp,  and  to  model  the  potential 
activity  described  by  Maxwell’s  equations  in  the  scalp  and  skull  layers.  This  operation  could  be  per¬ 
formed  without  imposing  an  arbitrary  source  model  because,  regardless  of  where  they  are  generated  in 
the  brain,  potentials  recorded  at  the  scalp  must  arise  from  volume  conduction  from  the  cortical  surface 
through  the  skull  and  scalp.  While  we  expect  this  method  to  produce  an  estimate  of  the  current  distri¬ 
bution  at  the  cortical  surface,  it  does  not  identify  the  number,  position,  or  orientations  of  sources.  (For 
instance,  the  skull  currents  resulting  from  a  tangential  dipole  source  in  the  cortex  might  still  be  diffi¬ 
cult  to  distinguish  from  that  caused  by  two  radial  dipole  sources.) 

For  this  application,  the  region  of  interest  is  limited  to  the  scalp  volume  and  skull  volume.  The  differ¬ 
ence  between  the  inner  surface  of  the  skull  and  the  outer  convexity  of  the  cortical  surface  is  assumed 
to  be  small  enough  to  be  neglected.  We  also  assume  that  the  potential  activity  in  the  region  of 
interest  is  described  by  Maxwell’s  equation,  that  the  boundary  condition  on  the  air-scalp  surface  is 
stated  by  equation  (2),  i.e.,  current  cannot  flow  out  of  the  scalp,  and  that  the  boundary  condition  on 
the  cortical  surface  is  stated  by  the  following  equation  (4)  which  describes  the  cortical  surface  poten¬ 
tials. 

u  =  G(  x,  y,  z  )  on  S2.  (4) 

We  also  assume  that  there  is  no  generating  source  within  the  scalp  or  skull  which  means  the  function  J 
in  equation  (1)  is  zero.  Although  of  course  there  is  no  general  solution  to  the  "inverse  problem," 
because  of  the  assumed  boundary  conditions  there  is  a  unique  solution  of  equation  (1),  which  may  be 
obtained  as  follows.  Applying  the  Finite  Element  Method  to  equation  (1)  with  boundary  conditions  in 
(2)  and  (4),  the  following  matrix  vector  relation  is  obtained  (Le  et  ah,  in  prep.) 

An  =  f.  (5) 

Here  u  is  a  vector  of  dimension  n,  which  is  a  numerical  approximation  to  the  analytical  potential  dis¬ 
tribution  function  u  in  equation  (1).  The  value  of  n  corresponds  to  the  total  number  of  vertices  on  all 
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the  finite  elements  in  the  region  of  interest.  Since  we  are  assuming  that  no  generating  sources  are 
present  in  the  skull  and  scalp,  J  =  0  and  therefore  f=0.  We  then  decompose  u  into  three  sets,  the 
potentials  which  correspond  to  the  electrodes  on  the  scalp,  the  potentials  which  correspond  to  the  corti¬ 
cal  surface,  and  the  potentials  in  the  rest  of  the  region,  and  denote  them  by  u1;  u3  and  u2  respectively. 
If  we  decompose  the  matrix  A  correspondingly,  then  equation  (5)  becomes: 


> 

> 

03 

f  X 

Ul 

A21  A  22  A03 

u2 

■^■31  A32  A33 

U3 

Solving  ux  with  respect  to  u3,  we  get 

(  An  —  A i o A 2o  A2i  )uj.  —  (  Ai2 A22^ A23  Aj3  )u3  (6) 

and  solving  u3  with  respect  to  ti!,  we  get 

A22^A23  )u3  —  (  A32A22^A2i  —  A31  )Uj .  (l) 

For  a  given  Gk,  u3  is  known.  Then  u*  can  be  computed  via  equation  (6)  and  the  residual  is  calculated 
from  uf  and  U(x,y,z)  which  correspond  to  the  predicted  and  measured  potentials,  respectively. 

S.  Estimating  conductivities  and  evaluating  deblurring  method:  Up  to  now,  estimates  of  conduc¬ 
tivity  for  the  scalp,  skull,  CSF  and  brain  have  been  published  (Geddes  &  Baker,  1967;  Hosek,  1970). 
We  have  attempted  to  estimate  skull  conductivity  by  injecting  small  currents  at  one  electrode  and 
recording  at  the  others  for  many  combinations  of  stimulating  and  recording  electrodes  (Gevins  and 
Doyle,  Unpublished  data).  This  has  failed,  however,  because  of  the  obvious  shunting  effect  of  the 
scalp.  We  are  currently  working  on  a  more  direct  approach  that  does  not  involve  applying  external 
stimuli  but  rather  uses  known  natural  sources  of  activity.  We  are  evaluating  the  method  by  compar¬ 
ing  deblurred  scalp  data  with  subdural  grid  data  recorded  from  the  same  patient. 

B.  Equivalent  Dipole  Modeling  of  Somatosensory  Stimulation 

We  are  just  beginning  to  apply  the  methods  described  above  to  functional-anatomical  localization. 
The  first  application  was  to  perform  equivalent  dipole  modeling  of  somatosensory  stimulation.  For  this 
purpose,  the  middle  fingers  of  the  left  and  right  hands,  and  the  right  index  finger  were  each  electrically 
stimulated  at  15  Hz  for  three  100-second  intervals.  Artifact-free  epochs  of  0.533  seconds  were  aver¬ 
aged,  and  Fourier-transformed  real  and  imaginary  components  of  the  15-Hz  response  were  averaged 
over  each  100-second  run.  Data  for  each  of  the  three  runs  were  then  combined  for  each  finger  for  cal¬ 
culation  of  the  "canonical"  phase,  i.e. ,  that  phase  angle  onto  which  the  projections  of  all  the  phasors 
yielded  the  greatest  squared  amplitude.  These  steady-state  amplitude  measures  were  used  to  construct 
topographical  maps  and  for  least-squares  fitting  of  equivalent  current  dipoles.  Preliminary  results  for 
the  first  subject  (Figure  11)  show  topographical  maps  of  the  Laplacian  of  the  evoked  responses  to 
stimulation  of  the  left  and  right  middle  fingers.  (Each  map  is  scaled  separately.  Amplitude  of  the 
right  finger  response  is  approximately  one  half  that  of  the  left.)  Figure  12  presents  the  result  of  fitting 
single  equivalent  dipoles  to  these  data  and  also  the  result  for  the  right  middle  finger.  Dipoles  are 
shown  with  respect  to  the  scalp  surface  and  eroded  brain  model  described  above.  Each  dipole  is 
represented  as  a  disk  with  it’s  center  on  the  dipole  location,  and  axis  in  the  dipole  direction.  Each 
dipole  appears  in  the  contralateral  hemisphere,  and  the  dipole  for  the  forefinger  is  located  slightly  more 
lateral  than  that  for  the  middle  finger,  consistent  with  the  known  locations  of  the  sensory  projection 
areas  and  other  physiological  source  localization  results  (Luders  et  al.,  1986;  Wood  et  al.,  1985). 
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CONCLUSION 

There  is  no  doubt  that  much  useful  information  lies  hidden  in  the  EEG.  We  are  optimistic  about 
advancing  our  technology  to  mine  this  hidden  information,  and  to  further  our  investigations  of  the 
neural  substrate  of  human  higher  cognitive  brain  functions. 
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FIGURE  LEGENDS 

Figure  1:  Diagram  of  124-channel  scalp  montage.  Electrode  names  are  based  on  10-20  system  with 
additional  letter  prefixes  and  numerical  suffixes  (see  text). 

Figure  2:  Subject  wearing  124-channel  EEG  hat. 

Figure  3:  Measurement  of  electrode  coordinates.  The  research  assistant  touches  each  electrode  with  a 
magnetic  position  sensor  while  the  subject  is  resting  his  head  on  a  chin  rest  built  into  a  head  support 
which  gently  restricts  head  movement  during  the  measurements.  The  3-D  coordinates  of  each  elec¬ 
trode  position  are  transmitted  to  the  data  collection  computer. 

Figure  4:  Graphic  display  of  measured  electrode  positions.  The  menu  at  the  left  lists  each  of  the  elec¬ 
trode  names.  An  initial  three  positions  are  used  to  determine  the  coordinate  system,  usually  T3,  T4 
and  Fpz.  A  circle,  representing  the  head  circumference,  is  then  drawn  around  these  three  positions. 
Subsequent  electrodes  are  shown  in  equidistant  projection,  with  electrodes  outside  the  circle  represent¬ 
ing  locations  lower  than  the  circumference  through  T3,  T4  and  Fpz. 

Figure  5:  Manscan*  functional-anatomical  neuroimaging  system. 

Figure  6:  Electrodes  schematically  displayed  as  small  cylinders,  at  the  actual  measured  postions,  on  a 
3-D  model  of  the  subject’s  head  constructed  from  his  MRIs. 

Figure  7:  Five  views  of  composite  MR  images.  Horizontal  slices  are  shown  in  yellow,  sagittal  in 
magenta  and  coronal  in  blue.  For  each  slice,  a  closed  contour  which  outlines  the  scalp  is  first  automat¬ 
ically  obtained  with  a  thresholding  algorithm.  The  composites  are  then  drawn  for  each  viewing  posi¬ 
tion  by  displaying  the  MR  intensity  values  contained  within  each  contour. 

Figure  8:  Composite  MR  images  showing  a  reconstructed  cortical  surface  (in  yellow)  produced  from  the 
automatically  traced  cortical  surface  contours.  Surface  is  relatively  smooth  because  it  was  recon¬ 
structed  from  a  standard  clinical  MRI  scan  with  5  mm  interslice  intervals.  Also  shown  are  images  con¬ 
tained  within  the  scalp  contour  for  one  sagittal  section  (magenta)  and  one  posterior  coronal  section 
(blue).  The  anterior  blue  image  is  the  coronal  data  contained  within  the  cortical  surface  contour. 

Figure  9:  (Left)  Midline  sagittal  MRI  obtained  with  TR=600  and  TE=20  ms.  (Right)  Output  of 
automatic  edge-detection  software.  The  lines  shown  correspond  to  locations  of  local  maxima  in  the 
gradient  of  the  image  intensity.  These  contours  pass  through  the  points  in  the  image  at  which  the 
local  average  intensity  is  changing  most  rapidly. 

Figure  10:  Automated  construction  of  finite  elements  within  brain,  skull  and  scalp  volumes,  with  alter¬ 
nating  elements  shown  in  red,  green  and  blue,  for  the  respective  tissue  type,  (a)  Tetrahedral  elements 
generated  in  an  outer  ring  in  brain,  (b)  An  entire  horizontal  slice  at  the  level  of  the  orbits,  (c)  All  slices 
superimposed  except  for  the  topmost  horizontal  slices. 

Figure  11:  Laplacian  of  124-channel  evoked  potentials  evoked  by  15-Hz  stimulation  of  the  middle 
finger  on  left  (top)  and  right  (middle)  hands  of  this  subject  mapped  onto  his  own  MRI-derived  scalp 
surface.  The  map  for  the  right  index  finger  is  shown  in  the  bottom  panel.  Color  scale  shows  zero  as 
white  and  the  maximum  as  magenta. 

Figure  12:  Integration  of  EEG  and  MRI  data  as  a  step  towards  anatomical-functional  localization.  The 
data  shown  in  Figure  11  were  used  to  compute  best-fitting  single  equivalent  dipoles  using  a  three- 
sphere  head  model.  The  figure  shows  top,  left  and  right  views  of  the  scalp  surface  reconstructed  from 
horizontal  MR  images.  The  voxel  brain  model  shown  in  light  blue  corresponds  to  four  stages  of  erosion 
of  3  mm  layers,  hence  its  shrunken  appearance.  The  dipole  for  right  index  finger  stimulation  (dark 
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blue)  is  located  most  left-laterally.  The  middle  finger  dipoles  (purple)  are  located  appropriately  in  con¬ 
tralateral  hemispheres. 
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V.  DISSEMINATION  OF  TECHNICAL  INFORMATION 


A.  Publications  dealing  with  methodology 

Gevins,  A.S.,  Brickett,  P.,  Costales,  B.,  Le,.  J.,  Reutter,  B.  (In  press)  Beyond  topographic  mapping: 
Towards  functional-anatomical  imaging  with  124-channel  EEGs  and  3-D  MRIs.  Brain  topography 
(see  Section  IV. D).  Application  of  the  method  of  Event-Related  Covariance  has  been  detailed  in 
EEG  clin.  Neurophys.,  7 4(1),  pp.  58-75,  and  in  Functional  Topography  of  the  Human  Brain,  (G. 
Pfurtscheller,  Editor)  pp.  99-116.  A  historical  discussion  of  be  tween-channel  correlation  measures  is 
presented  in  Chapter  6  of  the  Handbook  of  Electroencephalography  and  Clinical  Neurophysiology ,  Vol. 
1,  (A.S.  Gevins  and  A.  Remond,  Editors).  The  128  channel  recording  system,  and  latest  methods  of 
measuring  scalp  positions  and  registration  with  MRI  scams  has  been  presented  in  Brain  Topography, 
Dynamic  functional  topography  of  cognitive  tasks  (1989).  More  detailed  descriptions  are  planned. 
The  64-channel  system  and  earlier  methods  of  registration  were  presented  in  Dynamics  of  Sensory  and 
Cognitive  Processing  of  the  Brain  (E.  Basar,  Editor)  pp.  88-102.  Our  current  Laplacian  derivation 
procedure  is  described  in  Brain  Topography,  Dynamic  functional  topography  of  cognitive  tasks  (1989). 
Mathematical  methods  used  in  brain  modeling  are  described  in  Beyond  topographic  mapping  (In 
Press).  Pattern  recognition  methods  have  been  described  in  IEEE  ASSP  Trans.,  86(7),  pp.  1152- 
1161.  Tutorials  on  this  topic  were  previously  published  in  IEEE  Trans.  Patt.  Analy.  and  in  Chapter 
17  of  the  Handbook  of  Electroencephalography  and  Clinical  Neurophysiology,  Vol.  1.  We  are  also  try¬ 
ing  to  increase  awareness  about  the  need  for  technological  and  methodological  advances  in  mainstream 
psychophysiology  by  writing  detailed  analyses  of  the  findings  and  limitations  of  contemporary  ERP 
research,  and  the  technical  obstacles  which  must  be  overcome  to  bridge  those  limitations  (Gevins  and 
Cutillo,  Chapter  11  in  the  Handbook  of  Electroencephalography  and  Clinical  Neurophysiology,  Vol.  2 
(F.  Lopes  da  Silva,  W.  Storm  van  Leeuwen  and  A.  Remond,  Editors). 


B.  Presentations  at  meetings  1988-1989 

We  have  increased  the  number  and  types  of  meetings  at  which  we  present  our  results,  in  order  to 
disseminate  our  work  and  reach  researchers  with  different  specializations,  e.g.,  psychologists,  neuro¬ 
physiologists,  neurologists,  psychiatrists  and  biomedical  engineers.  A  partial  list  of  meetings  during 
1988  and  1989  to  which  the  PI  was  invited  to  speak  is  provided  here.  In  addition  to  these,  the  PI, 
Co-PI  and  other  EEGSL  scientists  have  made  invited  and  contributed  presentations  at  a  variety  of 
national  and  international  meetings. 

Invited  lectures  and  symposia  by  A.  Gevins,  1988  to  March,  1990 

Int.  Conf.  Pharmaco-EEG,  Japan,  1988  (Psychopharmacologists) 

Soc.  Psychophys.  Res.,  San  Francisco,  1988  (Psychophysiologists) 

Amer.  EEG  Soc.,  Sain  Diego,  1988  (Clinical  Neurophysiologists) 

Neurology  Grand  Rounds,  NINDS,  1988  (Neurologists  and  Psychophysiologists) 

IEEE  Conf.  Biomedical  Engineering,  New  Orleans,  1988  (Biomedical  Engineers) 

AFOSR  Cognitive  Science  Program  Review,  Colorado  Springs,  1988  (Cognitive  Psychologists) 

IEEE  Conf.  Biomedical  Engineering,  Seattle,  1989  (Biomedical  Engineers) 

Int.  Conf.  Topographic  EEG  Analysis,  Italy,  1989  (Clinical  Neurophysiologists) 

Int.  Summer  School  of  Brain  Research,  Netherlands,  1989  (Neuroscientsts) 

Int.  Cong.  EEG  and  Clin.  Neurophysiol.,  Brazil,  1990  (Neurophysiologists) 
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C.  Visitors  to  EEGSL  during  1988-1989 

G.  Fuller,  Psychophys.,  S.F. 

A.  Amochaev,  Psychophys.,  U.  Colorado 

B.  Davis,  Psychol.,  U.S.  Govt. 

B.  Lynch,  Neuropsych.,  Palo  Alto  VAMC 

R.  Mueller  and  V.  Calinshini,  Neurol.,  Pres.  Med.  Cntr.,  S.F. 

B.  Hudspeth,  Psychophys.,  Colorado 

R.  Thatcher,  Psychophys.,  U.  Maryland 

G.  Pfurtscheller,  Biomed.  Eng.,  Tech.  U.  Graz 

D.  Nelson,  Physics,  Tulane  University 
W.  Gersch,  Statistics,  U.  Hawaii 

P.  Stark,  Statistics,  U.C.  Berkeley 
J.  Moser,  Biomed.  Eng.  SRI 
M.  Melich,  Physics,  U.S.  Navy 

L.  Temoshok,  Psychol.,  Walter  Reed  Hospital 

B.  Englestad,  Nuc.  Med.,  UCSF 

A.  Chen,  Psychia.,  U.  Wash.  Seattle 
J.  Baribeau,  Psychophys.,  Concordia  U.,  Montreal 

H.  Hebenstreit,  Psychia.,  Munich. 

C.  Roy,  Neurol.,  UCSF 

M.  Toussaint,  Biomed.  Eng.,  U.  Paris 

S.  Klein,  Physiol.,  U.C.  Berkeley 

V.  Pollack,  Psychophys.,  U.  So.  California 
G.  Meshegan,  Physiol.,  U.  Texas  Dallas 
V.  Reuss,  Psychia.,  UCSF 

E.  Sutter,  Biomed.  Eng.,  Smith  Ketterwell  Inst.,  S.F. 

R.  Weiner,  Psychia.,  U.  No.  Carolina 

J.  Desmedt,  Neurol.,  Brussels 
M.  Scherg,  Biomed.  Eng.,  Munich 
R.  Cracco,  Neurol.,  SUNY  Brooklyn 

D.  Jewett,  Physiol.,  UCSF 
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Signs  of  Model  Making  by  the  Human  Brain 

A.S.  Gkvins 


1  Introduction 

The  brain  is  a  type  of  Rorschach  test  for  scientists.  Each  scientist  describes  the 
brain's  structure  or  function  with  the  language  and  imagery  of  his  or  her  own  par¬ 
ticular  trade,  be  it  mathematics,  physics,  computer  science,  neurophysiology,  or 
psychology.  But  these  descriptions  inevitably  fall  short  because  the  brain  itself  is 
its  own  metaphor.  With  this  in  mind.  I  am  going  to  speculate  that  recent  neuro- 
electric  measurements  we  made  from  people  performing  simple  cognitive  tasks 
may  be  interpreted  as  signs  of  the  internal  models  that  our  brains  use  to  interpret 
stimuli  and  take  action.  But  first.  I  should  review  the  methods  and  results. 


2  Signal  Processing  Methods 

2.1  New  Methods  for  Measuring  Neurocognitive  Neuroelectric  Patterns 

During  the  last  1 5  years,  my  olleagues  and  I  have  devoted  a  full-time  effort  to 
developing  mathematical  procedures  and  computer  systems  for  extracting  more 
specific  information  from  EEGs  (reviews  in  Gevins  and  Yeager  1972;  Gevins  et 
al.  1975;  Gevins  1980.  1984.  1987a.  b.  c.  d;  Gevins  and  Morgan  1986.  1988; 
Gevins  and  Bressler  1988).  So  far,  we  have  applied  these  methods  to  six  experi¬ 
ments  which  attempted  to  measure  human  higher  cognitive  functions  with  in¬ 
creasing  specificity  (Gevins  et  al.  1979a.  b.  c.  1981.  1985.  1987.  1988.  1989a.  b). 
For  each  set  of  two  experiments,  we  developed  a  novel  set  of  experimental  stimuli, 
analytic  procedures,  and  software  systems  to  overcome  the  shortcomings  of  the 
last  set. 

The  increasing  capability  of  laboratory  computers  has  been  the  engine  of  our 
progress.  Fifteen  years  ago.  I  naively  thought  that  if  I  could  only  apply  a  trillion 
arithmetic  operations  to  a  billion  bytes  of  neuroclectric  data.  I  could  find  out 
something  basic  about  how  human  brains  worked.  Now  that  this  capability  is 
routine.  I  am  eagerly  awaiting  the  100-trillion  operation.  10-billion-byte  experi¬ 
ment. 

The  need  for  this  much  computation  results  from  the  increasing  number  of 
scalp  EEG  channels,  as  well  as  the  increasingly  complicated  analyses  required  to 
measure  subtle  spatiotemporal  neurocognitive  signals  from  large  sets  of  single- 
trial  data.  Currently,  we  are  routinely  recording  64  EEG  channels,  and  analyzing 
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Fig.  5.  Schematic  diagram  of  how  the  previous  mo¬ 
ment's  internal  model  influences  the  current  moment's 
perception,  decision,  and  action,  and  how  the  feedback 
from  the  present  moment  influences  the  next  moment's 
internal  model.  Spauotemporal  neuroeiectnc  signs  of 
preparatory  and  feedback  networks  have  recently  been 
measured 


the  current  moment.  Based  on  these  models,  we  use  our  body's  effector  and  sen¬ 
sory  systems  to  actively  probe  the  environment  for  stimuli  relevant  to  our  needs 
( Fig.  5).  When  the  model  for  a  particular  task  is  incorrect,  as  with  incomplete  or 
inappropriate  preparation,  subsequent  performance  is  usually  incorrect.  When 
feedback  follows  action,  the  brain  model  updates. 

1  interpret  the  preparatory  and  feedback  "networks."  aspects  of  which  we 
have  measured,  as  signs  of  these  situation-specific  internal  models.  With  further 
development,  we  can  expect  to  have  increasingly  detailed  direct  measurements  of 
how  our  experience  of  reality  is  synthesized  in  the  brain. 


5  Summary 


Using  state-of-the-art  recording  and  signal  processing  procedures,  we  measured 
spatiotemporal  neuroeiectnc  patterns  as  subjects  prepared  to  produce  a  precise 
finger  pressure  to  a  visual  stimulus,  as  they  made  the  response,  and  as  they  re¬ 
ceived  feedback  information  about  the  accuracy  of  their  response.  For  each  of 
these  three  situations,  we  calculated  ERC  patterns  between  pairs  of  electrodes. 

ERCs  that  preceded  accurate  left-  or  right-hand  performances  overlay  the  left 
frontal  cortex,  and  'he  motor  and  parietal  cortices  contralateral  to  the  performing 
hand.  In  addition,  strong  ERCs  overlying  midline  motor  and  premotor  cortices 
preceded  left-hand  performance.  This  "preparatory  network"  comprised  distinct 
cognitive,  somesthetic-motor.  and  integrative  motor  cortical  areas.  We  used  its 
presence  or  absence  to  predict  the  accuracy  of  responses  that  began  0.5-1  s 
later. 

Similarly,  we  measured  a  "feedback  network"  in  an  interval  spanning  an  early 
P300  peak.  Signals  from  bilateral  and  midline  frontal  sites  were  involved  in  pat¬ 
terns  related  to  inaccurate  feedback,  whereas  the  left  and  midline  frontal  sites 
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3.3  The  Fourth  Experiment  (Gevins  et  al.  1988) 

After  learning  and  practicing  a  difficult  visuomotor-memory  task  ( VMMT)  until 
their  performance  was  stable,  each  of  five  healthy,  right-handed,  male  subjects  re¬ 
turned  to  the  laboratory  the  next  morning  and  performed  several  hundred  trials 
of  the  task.  This  task  required  subjects  to  remember  two  continuously  changing 
numbers,  in  the  presence  of  numeric  distractors.  and  produce  precise  finger  pres¬ 
sures.  Each  trial  consisted  of  a  warning  symbol,  followed  by  a  single-diait  visual 
stimulus  to  be  remembered,  followed  by  the  subject's  finger-pressure  response  to 
the  stimulus  number  presented  two  trials  ago.  followed  b>  a  two-digit  feedback 
number  indicating  the  accuracy  of  the  response.  For  example,  if  the  stimulus 
numbers  in  five  successive  trials  were  8.  6.  !.  9.  4.  the  correct  response  would  be 
a  pressure  of  0.8  kg  when  seeing  the  1.  0.6  kg  for  the  9.  and  0.1  kg  for  the  4.  To 
increase  the  task  difficulty,  subjects  were  required  to  withhold  their  response  on 
a  random  20“  o  of  the  trials.  These  "no-response  catch  trials”  were  trials  in  which 
the  current  stimulus  number  was  identical  to  the  stimulus  two  trials  ago.  After 
the  experiment,  we  divided  each  subject's  responses  into  "relatively  accurate”  and 
"relatively  inaccurate”  trials,  depending  on  whether  the  error  of  an  individual 
trials  was  less  than  or  greater  than  that  subject's  mean  accuracy  over  the  session. 
Accurate  and  inaccurate  trial  sets  were  randomly  distributed  across  the  session. 

EEGs  were  recorded  with  either  33  or  51  channels  set  in  a  nylon  mesh  cap. 
Vertical  and  horizontal  eye  movements  were  also  recorded,  as  were  the  respond- 
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Fig.  4  ERCs  preceding  accurate  and  inaccurate  performance  of  a  task  that  required  subjects  io 
produce  precise  finger  pressures  proportional  to  numbers  held  in  memory.  Significant  (R<0.05) 
ERC  patterns  during  a  500-ms  interval  when  five  subjects  were  remembering  two  numbers  and 
awaiting  a  new  number  to  be  remembered.  The  ERC  patlern  preceding  accurate  performance 
is  stronger,  and  differs  from  the  inaccurate  patlern  principally  in  the  posterior  channels 
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7.  Using  the  magnitude  of  the  maximum  cross-covariance  function  and  its  lag 
time  to  characterize  the  ERC 

8.  Estimating  significance  of  ERCs  by  the  standard  deviation  of  the  “noise" 
ERC  estimated  from  randomized  data 

9.  Graphing  the  most  significant  ERCs  in  each  interval 

10.  Statistically  comparing  ERC  distributions  between  experimental  conditions. 


2.3  Validating  ERC  Measurements 

The  current  ERC  analysis  has  been  applied  to  data  recorded  from  several  experi¬ 
ments.  The  validity  of  the  method  is  demonstrated  in  analy  sis  of  visual  stimulus 
processing  and  response  execution  intervals  of  a  visuomotor  task  (Gevins  et  al. 
19S7).  During  the  period  of  visual  processing.  ERC  patterns  involved  posterior 
sues  followed  by  anterior  parietal  and  premotor  sites.  This  finding  was  consistent 
with  neuroanatomical  considerations  and  clinical  neuropsychological  studies. 
Likewise.  ERC  patterns  for  movement-related  intervals  corresponded  to  prior 
functional  neuroanatomical  studies  of  primates  and  patients  with  intracerebral 


Fie.  1 .  Most  significant  (top  one  standard  deviation)  ERC  patterns  at  the  peak  of  the  finger  pres¬ 
sure  for  right-  and  left-hand  trials  from  seven  nght-handed  subjects,  superimposed  on  maps  of 
Lapiacian  response  potential  amplitude  The  thickness  of  an  ERC  line  is  proportional  to  the  neg¬ 
ative  log  of  us  significance  (from  0.05  to  0  00005).  The  color  of  a  line  shows  the  lag  time  (ms 
delay)  between  sites.  Note  that  the  anterior  midline  precentral  taCz)  electrode  is  the  focus  of  all 
ERCs.  The  patterns  are  distinctly  lateralized  according  to  responding  hand.  The  sign  of  the  aCz 
ERCs  is  positive  (yellow)  for  lateral  frontal,  and  negative  (tan)  for  lateral  central  and  anterior 
parietal  electrodes 


Fig.  2a.b.  Significant  (P  <  0.05)  prestimulus  ERC  patterns  superimposed  on  maps  of  Laplacian 
CN'V  amplitude.  Same  subjects  and  task  as  Fig. !.  Measurements  are  from  an  interval  of  500- 
87 5  ms  after  the  cue  tor  subsequently  accurate  and  inaccurate  right-  la)  and  left-hand  (b)  visuo- 
motor  task  performance  by  seven  right-handed  men.  The  thickness  of  a  covariance  line  is  pro¬ 
portional  to  its  significance  (from  0.05  to  0.005).  A  violet  line  indicates  the  covariance  is  positive, 
while  a  blue  line  is  negative.  ERCs  involving  left  frontal  and  appropriately  contralateral  central 
and  parietal  electrode  sites  are  prominent  in  patterns  for  subsequently  accurate  performance  of 
both  hands  The  magnitude  and  number  of  ERCs  are  greater  preceding  subsequently  inaccurate 
left-hand  performance  by  these  right-handed  subjects,  and  are  more  widely  distributed  compared 
with  the  left-hand  accurate  pattern.  For  the  nght-hand.  fewer  and  weaker  ERCs  characterize 
subsequently  inaccurate  performance.  The  amplitude  maps  are  very  similar  for  the  four  condi¬ 
tions.  and  do  not  indicate  any  of  the  specific  differences  evident  in  the  covariance  patterns 
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was  followed  by  a  visual  numeric  stimulus  (number  1-9)  indicating  that  a  pressure 
of  0.1  to  0.9  kg  should  be  made  with  the  index  finger  of  the  previously  indicated 
hand.  A  two-digit  number,  presented  1  s  after  the  peak  of  the  response  pressure, 
provided  feedback  that  indicated  the  subject's  exact  pressure.  On  a  random  20°o 
of  the  trials,  the  stimulus  number  was  slanted  in  the  opposite  direction  to  the  cue: 
subjects  were  to  withhold  their  responses  on  these  "catch  trials."  The  next  trial 
followed  1  s  after  disappearance  of  the  feedback.  Each  subject  performed  several 
hundred  trials,  with  rest  breaks  as  needed. 

Twenty-six  channels  of  EEG  data,  as  well  as  vertical  and  horizontal  eye  move¬ 
ments  and  flexor  digiton  muscle  activity  from  both  arms,  were  recorded.  All  sin¬ 
gle-trial  EEG  data  were  screened  for  eye  movement,  muscle  potential,  and  other 
artifacts.  Contaminated  data  were  discarded. 

Intervals  used  for  ERC  analysis  centered  on  major  event-related  potential 
peaks.  ERCs  were  computed  between  each  of  the  120  pairwise  combinations  of 
the  16  nonpenpheral  channels.  Intervals  were  set  from  500  ms  before  the  cue  to 
500  ms  after  the  feedback. 

We  first  calculated  the  mean  error  (deviation  from  the  required  finger  pres¬ 
sure)  over  all  trials  from  the  recording  session.  Individual  trials  were  then  classi¬ 
fied  as  accurate  (trial  error  less  than  mean  error)  or  inaccurate  (error  greater  than 
mean  error). 

ERC  patterns  during  a  375-ms  interval  centered  687  ms  postcue  (spanning  the 
late  contingent  negative  variation-CNV).  regardless  of  subsequent  accuracy,  in¬ 
volved  left  prefrontal  sites,  as  well  as  appropriately  lateralized  central  and  parietal 
sites.  Inaccurate  performance  by  the  right  hand  was  preceded  by  a  very  simple 
pattern  (Fig.  2a),  while  inaccurate  performance  by  the  left  hand  was  preceded  by 
a  complex,  spatially  diffuse  pattern  (Fig.  2  b).  The  relative  lack  of  ERCs  preceding 
inaccurate  right-hand  performance  may  simply  reflect  inattention  on  those  trials, 
while  the  strong  and  complex  patterns  preceding  inaccurate  performance  with  the 
left  hand  may  reflect  effortful,  but  inappropriate,  preparation  by  the  right- 
handed  subjects. 

ERC  patterns  related  to  feedback  about  accurate  and  inaccurate  per¬ 
formances  were  similar  immediately  after  the  onset  of  feedback,  but  began  to  dif¬ 
fer  in  an  interval  centered  at  281  ms.  that  spanned  the  early  P3  peak  (P3E)  (Fig.  3). 
The  ERC  patterns  for  feedback  to  accurate  performance  by  the  two  bands  were 
very  similar  (bootstrap  correlation  =  0.91  —  0.01 ).  involving  midlinc  anterocen- 
tral.  central,  anteroparietal.  parietal,  and  antero-occipital  sites,  left  anteroparietal 
and  anterocentral  sites,  and  right  parietal,  anteroparietal.  anterocentral.  and 
frontal  sites.  These  accurate  patterns  involved  many  long-delay  (32-79  ms) 
ERCs.  The  waveforms  of  the  frontal  and  anterocentral  sites  consistentlv  lagged 
those  of  more  posterior  sites.  For  feedback  to  inaccurate  performance,  patterns 
for  both  hands  were  also  very  similar  (bootstrap  correlation  =  0.90  ±0.02).  and 
involved  most  of  the  same  sites  as  the  accurate  patterns,  with  the  striking  excep¬ 
tion  of  the  left  and  midline  frontal  sites.  Again,  frontal  waveforms  lagged  those 
of  the  more  posterior  sites  with  which  they  covaried.  There  were  even  more  long- 
delav  ERCs  than  in  the  accurate  patterns. 

We  suggest  that  our  prestimulus  ERC  patterns  characterize  a  distributed  pre¬ 
paratory  neural  set  that  is  related  to  the  accuracy  of  subsequent  task  per- 


Signs  of  Model  Making  by  the  Human  Brain 


411 


electrodes  (Fig.  1):  the  midline  precentral  electrode  that  overlays  the  premotor 
and  supplementary  motor  cortices  was  the  focus  of  all  movement-related  ERC 
patterns,  and  the  other  most  significant  ERCs  involved  pre-  and  postcentral  sites 
that  were  appropriate  contralateral  to  the  responding  hand.  Moreover,  the  posi¬ 
tive  sign  of  the  ERCs  between  midline  precentral  and  lateralized  precentral  sites 
and  the  negative  sign  between  midline  precentral  and  postcentral  sites  are  consis¬ 
tent  with  the  sharply  focused  current  sources  and  sinks  spanning  the  hand  areas 
of  somatosensory  and  motor  cortices. 

While  these  ERC  patterns  appear  to  reflect  neatly  the  functional  coordination 
of  the  immediately  underlying  cortical  areas,  we  must  again  emphasize  that  their 
actual  neural  sources  are.  in  fact,  not  yet  known.  Some  caution  should  therefore 
be  exercised  in  interpreting  the  patterns  too  literally. 


3  Preparation  and  Feedback 

During  Simple  Visuomotor  Judgment  Tasks 

3.1  The  First  Two  Experiments 

In  a  series  of  four  experiments,  we  attempted  to  characterize  distributed  networks 
of  cortical  function  during  simple  tasks.  In  the  first  experiment  (Gevins  et  al. 
1981).  we  measured  event-related  cross-correlations  between  91  pairwise  combi¬ 
nations  of  recordings  from  1 5  electrodes  during  performance  of  two  simple  tasks, 
one  requiring  a  spatial  judgment  and  the  other  a  numeric  judgment.  A  cue  before 
each  trial  indicated  the  type  of  task  to  be  performed.  The  cross-correlation  pat¬ 
terns  of  the  two  tasks  differed  during  the  last  300  ms  of  the  cue  interval.  These 
differences,  which  included  midline  frontal,  parietal,  and  occipital  electrode  sites, 
suggested  that  neural  networks  associated  with  preparation  were  both  task  spe¬ 
cific  and  spatially  detailed.  These  "preparatory  networks"  were  better  visualized 
in  a  second  experiment,  which  extracted  event-related  cross-correlations  that  dis¬ 
tinguished  two  similar  spatial  tasks  (Gevins  et  al.  1983.  1985).  Spatially  distrib¬ 
uted.  task-specific  "preparatory  networks"  were  found  again.  This  time,  the  mid¬ 
line  frontal  (Fz)  electrode  was  the  only  site  that  showed  a  task-dependent  differ¬ 
ence  during  the  last  300  ms  of  the  cue  interval. 


3.2  The  Third  Experiment 

We  designed  a  third  experiment  based  on  these  encouraging  results.  We  wished 
to  manipulate  preparatory  sets  more  explicitly,  and  to  investigate  feedback  about 
performance  accuracy  (Gevins  et  al.  1987.  1989  b).  Along  with  a  new  experimen¬ 
tal  design,  we  had  to  develop  new  signal  processing  procedures  that  more  robustly 
measured  event-related  covariance  between  electrode  sites. 

Seven  healthy,  right-handed,  male  adults  participated  in  this  study.  A  visual 
cue.  slanted  to  the  right  or  to  the  left,  prompted  the  subject  to  prepare  to  make 
a  response  pressure  with  the  right  or  left  index  finger.  One  second  later,  the  cue 
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Fig.  3.  Most  significant  (top  two  largest  standard  deviations)  early  P3  (P3E)  feedback  ERC  pat¬ 
terns  for  feedback  to  accurate  and  inaccurate  performance  by  the  right  hand.  Same  subjects  and 
task  as  Fig.  1  and  2.  Same  scale  as  Fig.  I .  Both  the  ERCs  and  amplitude  maps  are  derived  from 
a  187-ms-wide  interv  al,  centered  at  281  ms  after  feedback  onset,  on  seven-subject  averaged  wave¬ 
forms.  A  major  difference  between  accurate  and  inaccurate  patterns  occurs  at  the  left  and  midline 
frontal  sites,  which  are  only  involved  in  the  inaccurate  patterns.  The  involvement  of  these  sites 
may  reflect  the  fact  that  greater  processing  is  required  following  inaccurate  performance  in  order 
to  update  the  preparatory  set  for  the  next  tnal 


formance.  This  network  appears  to  involve  distinctive  cognitive  (frontal),  integra- 
tive-motor  (midline  precentral),  and  lateralized  somesthetic-motor  (central  and 
parietal)  components.  The  involvement  of  the  left  frontal  site  is  consistent  with 
clinical  findings  that  preparatory  sets  are  synthesized  and  integrated  in  prefrontal 
cortical  areas,  and  with  experimental  and  clinical  evidence  indicating  involvement 
of  the  left  dorsolateral  prefrontal  coitex  in  delayed  response  tasks.  A  midline 
anterocentral  integrative  motor  component  is  consistent  with  known  involvement 
of  premotor  and  supplementary  motor  areas  in  initiating  motor  responses.  The 
finding  of  an  appropriately  lateralized  central  and  parietal  component  is  consis¬ 
tent  with  evidence  from  primates  and  humans  for  neuronal  firing  in  motor  and 
somatosensory  cortices  prior  to  motor  responses. 

Since  ERC  feedback  patterns  of  accurate  or  inaccurate  performance  (involv¬ 
ing  either  hand)  were  more  similar  than  those  between  accurate  and  inaccurate 
patterns  for  one  hand,  we  infer  that  the  feedback  patterns  were  related  more  to 
performance  accuracy  than  to  the  hand  used.  The  fact  that  ERC  patterns  follow¬ 
ing  discontinuing  feddback  involved  more  frontal  sites  than  did  patterns  follow¬ 
ing  confirming  feedback  is  consistent  with  the  idea  that  greater  resetting  of  per¬ 
formance-related  neural  systems  is  required  following  disconfirming  feedback. 
Likewise,  the  front  focus  of  these  differences  is  consistent  with  the  importance  of 
the  frontal  lobes  in  the  integration  of  sensors'  and  motor  activities  (Fuster  1984; 
Stuss  and  Benson  1986). 
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about  25  200-400  ms  intervals  spanning  each  4-  to  6-s  task  trial.  Each  trial  con¬ 
sists  of  a  cue.  stimulus,  response,  and  feedback  about  performance  accuracy:  the 
intervals  are  time  registered  to  the  evoked  potential  peaks  elicited  by  these  events. 
(In  February.  1988.  we  completed  the  expansion  to  128  channels). 

Much  of  our  effort  has  dealt  with  developing  various  spatial  and  temporal  sig¬ 
nal  enhancement  techniques,  of  which  the  method  of  event-related  covariance 
(ERC)  is  the  latest.  This  method  provides  a  window,  albeit  limited,  on  the  dy¬ 
namic  functional  neural  networks  which  produce  simple  goal-directed  behaviors. 
ERCs  measure  two  features  of  short  segments  of  event-related  potentials  l  ERPsi: 
the  similarity  of  waveform  and  the  timing  between  electrodes.  ERC  analysis  is 
based  on  animal  studies  which  showed  that  when  a  brain  region  becomes  involved 
in  stimulus-related  processing,  synchronization  of  a  subset  of  neurons  in  that  re¬ 
gion  is  manifested  as  a  change  in  the  form  of  the  region's  extracellularly  recorded, 
low-frequency  macropotentials  (reviewed  in  Gevins  and  Bressleri.  Covariance 
and  correlation  can  measure  waveform  similarity  and  timing  of  macropotentials 
from  different  areas  of  the  brain,  so  these  measurement  techniques  may  charac¬ 
terize  the  spatial  organization  of  coordinated  functional  activ  ity  of  the  areas  in¬ 
volved  in  a  goal-directed  behavior.  Of  course,  recording  at  the  scalp,  rather  than 
within  the  brain,  introduces  some  complications  into  this  simple  idea.  Although 
for  the  specific  cases  we  have  measured  ERCs  seem  to  refect  the  coactivation  of 
underlying  cortical  areas,  the  net  effect  of  these  complications  is  that  we  cannot 
yet  definitively  say  whether  this  is  true  in  general.  Our  current  research  is  focused 
on  resolving  this  issue  in  some  important  paradigmatic  cases. 

2.2  Computing  Event-Related  Covariances  (ERCs) 

There  are  ten  steps  in  computing  ERCs  (Gevins  et  al.  1 987;  Gevins  and  Bressler): 
the  tlrst  four  reduce  spatial  smearing  and  select  intervals  and  trials  that  contain 
task-related  information  in  order  to  enhance  the  signal-to-noise  ratio  and  reduce 
the  amount  of  data.  The  next  six  steps  measure  ERCs  on  bandpass-filtered,  en¬ 
hanced  averages  computed  from  the  reduced  data  set. 

The  ten  steps  involve: 

1.  Recording  at  least  10u  trials  of  each  task  condition  of  the  experiment,  using 
at  least  24  electrodes 

2.  Applying  the  Laplacian  operator  to  the  potential  distribution  of  each  elec¬ 
trode  not  located  at  the  edge  of  the  scalp,  in  order  to  remove  the  effect  of 
choice  of  reference  electrode  and  reduce  spatial  blur  from  volume  conduction 
through  the  skull  and  scalp 

3.  Removing  data  that  were  contaminated  with  artifact 

4.  Finding  trials  with  consistent  event-reiated  signals,  and  computing  enhanced 
averages  from  these  trials  (optional) 

5.  Selecting  digital  bandpass  niters  and  intervals  for  analysis  by  examining 
ERPs.  amplitude  distribution  maps,  and  (optional)  Wigner  time-frequency 
distributions 

6.  Computing  the  multilag  cross-covariance  functions  between  all  pairwise  (by 
channel)  combinations  of  the  interval’s  enhanced  averages 
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ing  flexor  digitori  muscle  potentials,  electrocardiogram,  and  respiration.  Three- 
axis  magnetic  resonance  image  scans  were  made  of  three  of  the  five  subjects. 

Grand-average  (over  the  five  subjects)  ERPs  were  time-locked  to  the  presen¬ 
tation  of  the  numeric  stimulus.  Spatiotemporal  neuroelectric  patterns  were  then 
quantified  by  measuring  ERCs  between  all  153  pairwise  combinations  of  the  18 
nonpenpheral  electrodes  in  the  common  subset  of  33  channels.  ERCs  were  mea¬ 
sured  on  the  grand-average  ERPs  across  a  500-ms  interval,  centered  312  ms  be¬ 
fore  the  numeric  stimulus.  During  this  interval,  subjects  were  maintaining  the  last 
two  visually  presented  numbers  in  working  memory,  and  preparing  for  the  next 
stimulus. 

Complex,  but  distinct  ERC  patterns  characterized  the  subsequently  accurate 
and  inaccurate  data  sets  (Fig. 4).  The  patterns  differed  both  in  overall  magnitude 
(/  =  3.3.  df—  48.  RcO.Ol )  and  in  topographic  distribution  (bootstrap  correlation 
=  0.37—0.14).  the  accurate  preparation  having  stronger  ERCs  overall.  Midline 
central  (607).  midline  precentral  (377).  left  posteroparietal  (257).  and  right  parie¬ 
tal  (204)  electrodes  had  the  largest  summated  ERCs  in  the  accurate  data  set.  For 
the  inaccurate  data  set.  the  four  largest  summated  ERCs  were  at  midline  central 
(569).  midline  precentral  (392).  left  frontal  (322).  and  right  anterocentral  (316). 
Deferring  discussion  of  the  implications  of  the  shift  from  posterior  to  anterior 
ERC  foci,  we  note  here  that  the  ERC  differences  preceding  accurate  and  inac¬ 
curate  performance  of  each  trial  of  the  visuomotor  memory  task  provide  another 
example  of  the  importance  of  task-specific,  preparatory  cortical  networks  in  cor¬ 
rect  action. 


4  Conclusions 

4.1  Methodological 

Event-related  covariance  analysis  seems  to  represent  a  useful  improvement  over 
topographic  brain  electrical  activity  maps  by  showing  what  may  prove  to  be  signs 
of  functional  interrelations  between  areas  of  the  cerebral  cortex.  Furthermore, 
differences  between  conditions,  which  are  evident  in  ERC  patterns  of  preparatory 
sets,  are  not  apparent  on  the  corresponding  topographic  maps  (Fig.  2)  (Gevins  et 
al.  1989a).  In  our  current  research  we  are  trying  to  determine  the  distributed 
sources  in  the  brain  that  produce  the  ERC  patterns  at  the  scalp. 


4.2  Brain  Model 

Although  it  is  convenient  to  describe  brains  using  the  language  of  computer 
science,  our  brains  are  not  actually  like  computers.  They  are  not  stimulus-re¬ 
sponse  machines  that  passively  await  stimuli,  process  them  according  to  fixed  al¬ 
gorithms.  and  then  make  responses.  Rather,  we  know  from  introspection,  obser¬ 
vation.  and  inference  that  our  brains  continuously  maintain  and  update  detailed 
representations  (models)  of  what  we  imagine  our  self-  and  world-states  to  be  at 
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To  represent  the  distributed  corneal  network  in¬ 
volved  in  goal-aireaed  behaviors.  it  is  necessary  to 
quantity  event-related  processing  in,  as  well  as  rela¬ 
tions  between,  the  funcaonai  centers  of  the  network. 
This  paper  discusses  the  idea  that  appropriately 
processed,  scaip-recorded  event-reiated  potentials 
can  index  the  activity  of  corneal  regions  involved  in 
cognitive  task  performance.  It  also  presents  1  set  of 
procedures,  called  Event-Related  Covariance 
(ERO  Analysis,  that  we  use  to  measure  patterns  of 
statistical  relationship  between  tteuroeieeme  ame- 
senes  recorded  at  different  scalp  sites.  Applying 
ERC  .Analysis  .o  simple  cognitive  tasks  produced 
dear-cut  results.  These  were  consistent  with  prior 
neuropsychological  models  of  the  rapidly  shifting 
cortical  network  accompanying  expectancy,  stimu¬ 
lus  registration  and  feature  extraction,  response 
preparation  and  execution,  and  "updating*  to  feed¬ 
back  about  response  accuracy. 

In  studies  of  higher  cognitive  functions  of  the 
human  brain,  hi  ere  currently  is  a  renaissance 
of  interest  in  measuring  spatial  aspeca  of 
brain  activity.  One  sign  of  this  is  the  reawak¬ 
ened  concern  with  dipole  source  localization 
in  the  brain.  Yet.  if  this  resurgence  is  to  nour¬ 
ish.  advances  in  psychophysiology  and  the 
technologies  of  brain  signal  measurement  and 
analysis  will  have  to  be  utilized  with  a  more 
sophisticated  approach.  While  the  localization 
of  single  equivaiem-dipole  generators  may  be 
quite  informative  in  the  case  of  sensory  and 
motor  events,  it  is  of  dubious  uulity  in  the 
case  of  higher  cognitive  processes.  In  fac,  it  is 
well  known  from  clinical  studies  that  even 
simple  cognitive  functions  must  require  inte¬ 
grated  processing  in  a  nerwork  involving  a 
numoer  of  distributed,  specialized  coracai 
areas  (Mesulara.  1981).  So,  although  it  is  con¬ 
venient,  it  may  not  always  be  a  good  idea  to 


compact  ail  the  data  into  a  single  equivalent 
dipole. 

Rather,  it  seems  more  desirable  to  charac¬ 
terize  the  dynamic  topology  of  this  distributed 
nerwork  to  determine  which  areas  are  active  at 
any  instant  of  time  and  which  areas  are  statis¬ 
tically  related  to  each  other.  This  is  an  idea 
that  has  been  evolving  over  the  past  four  dec¬ 
ades  (Waiter  i.  Shipton.  1951 :  3ariow  <t  Braz¬ 
ier.  1954;  Adey  et  ai_,  1961 :  John  et  ah,  1973: 
Callaway  k.  Harris.  1974;  Livanov,  1977; 
Tucker  et  aL.  1986;  Gevins  et  xL,  1981.  1983, 
1987;  Gevins,  1987a;  Bressler,  1987a,b).  Our 
current  approach  is  based  on  the  hypothesis 
that  when  regions  of  the  brain  are  functionally 
related,  their  event-related  potential  (ERP) 
components  are  related  in  shape  and  line  up 
in  time,  perhaps  with  some  delay.  The  idea  is 
that  die  ERP  waveform  delineates  the  course 
of  event-related  mass  neural  activity  of  a  pop¬ 
ulation.  so  that  if  two  populations  are  func¬ 
tionally  related,  their  ERPs  should  line  up.  If 
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jo,  and  if  the  relations  are  linear,  this  could  be 
measured  by  computing  the  lagged  covariance 
between  the  ERPs.  or  portions  of  the  ERPs, 
from  different  regions— a  measure  we  call 
event-related  covariances  (ERCs).  As  we  shall 
see  later,  initial  results  of  this  approach  have 
been  quite  promising.  We  must  caution,  how¬ 
ever,  that  the  hypothesis  that  ERCs  measure 
functional  relationships  between  cortical 
areas  has  not  yet  been  proven. 

The  measurement  of  ERCs  is  part  of  a  set 
of  signal  processing  procedures  called  *N eu¬ 
ro  cognitive  Pattern  Analysis”  (NC?  Analysis) 
that  -e  have  developed  to  extract  task-related 
spatiocemporal  patterns  from  the  unrelated 
electrical  activity  of  the  brain  (Figure  1).  In 
the  past  10  years,  these  procedures  have  be¬ 
come  increasingly  sophisticated  as  the  capa¬ 
bilities  of  computers  have  expanded.  They 
now  measure  spauotemporal  task-related  pro¬ 
cesses  from  up  to  64  scalp  electrodes  in  each 
of  up  to  25  fracdon-of-a-second  intervals 
spanning  a  4—6  second  period  extending  from 
before  a  cue,  through  stimulus  and  response, 
to  presentation  of  feedback  about  perform¬ 
ance  accuracy.  This  paper  describes  first  the 
basis  for  our  approach  to  characterizing  the 
functional  topography  of  the  brain,  and  then 
the  current  state  of  procedures  that  we  are 
developing  to  carry  out  that  characterization. 

Neurophysiological  Basis 
for  the  Measurement 
of  Functional  Topography 

The  representation  of  a eo cortex  as  a  distrib¬ 
uted  neural  network  of  interacting  functional 
centers  is  a  legacy  of  the  Sherringtoman  tradi¬ 
tion  (Sherrington.  1906).  The  pioneering  work 
of  Freeman  (1975)  has  been  crudai  in  recon¬ 
ciling  this  traditional  view  with  more  recent 
concepts  of  neural  networks  of  individual 
neurons,  by  redefining  the  funcaonai  center 
as  a  cooperative  domain  of  interconnected 
neuronal  populations,  rather  than  simply  an 
anatomical  pool  of  neurons.  Freeman  has 
demonstrated  a  systematic  order  between 
pulse  probaoiiities  of  individual  neurons  and 
macroscopic  forms  of  cooperauve  neural  ac¬ 
tivity  'macropotenuais),  showing  that  rmcro- 


potentials  reflea  the  emergence  of  dynamic 
seif-organizing  order  in  neuronal  populations, 
lasting  from  tens  to  hundreds  of  milliseconds 
(Freeman,  1987).  These  ideas  are  consistent 
with  the  findings  of  Eiui  (1972),  who  con¬ 
cluded  that  extra ceiluiariy  recorded  cortical 
macropotentials  result  from  summation  of 
dendritic  activity  of  the  synchronized  portion 
of  a  neuronal  population,  and  of  Pttsche  a  al. 
(1984),  who  demonstrated  the  importance  of 
the  dose  relation  between  macro  potentials 
and  local  cortical  architectonics. 

In  laying  the  foundation  for  the  concept  of 
distributed  cortical  networks,  studies  on  the 
emergent  nature  of  macropotentiais  have  been 
complemented  by  animal  experiments  focus¬ 
ing  on  field  potential  co-synchronizanon 
(Adey  et  ai_  1961;  Boudreau,  1966;  Livanov, 
1977).  Dumenko  (1970)  found  in  dogs  that 
correlation  between  extracellular  potentials  of 
visual  and  motor  cortices  increased  -with  con¬ 
ditioning  to  a  visual  stimulus  and  was  highe.t 
at  the  time  of  the  conditioned  stimulus  and  re¬ 
sponse.  The  finding  that  high  correlation  was 
specific  to  particular  regions  of  motor  and  vi¬ 
sual  cortices  was  similar  to  the  observation  of 
Bressier  (1987a,b)  that  high  correlation  be¬ 
tween  olfacdjry  bulb  and  cortex  in  rabbits  was 
specific  to  particular  regions  of  each.  Mea¬ 
sures  of  similarity  between  field  potentials 
have  also  been  applied  to  averaged  event- 
reiated  potentials.  John  et  al.  (1973)  measured 
the  reiatedness  of  waveforms  recorded  from 
diiTerent  conditioned  responses  and  demon¬ 
strated  differential  generalization  of  neural  ac- 
avity  based  on  waveform  similarity. 

Given  this  view  of  distributed  functional 
corneal  networks,  and  given  the  sensitivity  of 
ERPs  to  rapid  changes  in  brain  state  asso¬ 
ciated  with  performance  of  complex  behav¬ 
ioral  tasks,  we  hypothesize  that  appropriately 
processed  ERPs  may  re  flea  the  aenvanon  of 
rapidly-shifting,  widely  distributed  functional 
aggregates  of  underlying  neuronal  popula¬ 
tions.  In  the  tradition  of  Lashley  (1958)  and 
John  (1967),  we  have  sougnt  to  characterize 
the  “functional  topo^rarny"  of  distributed 
cortical  networks  by  looking  for  consistent 
statistical  relations  between  fieid  potentials 
recorded  over  different  areas  of  conex  during 
performance  of  highly  controlled  cognitive 
tasks. 
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PATTERN  RECOGNITION  SYSTEM 

RECORD  DATA 


SPAROTEMPCRAL  SJCxSAL  ENHANCEMENT 


r 


>  Reduce  nuw  conduction  iminng 
with  Lioiaaan  operator  or  spatial 
Mconvcuun 

•  Uu  a  nortnm.  layered  -  network  sartem 
rKgpnum  itgomnm  to  oetertnne  tuosets 
of  tnaia  nterveia  i no  cnamea 
witn  event  related  information 


EXTRACT  SPAROTEMPCRAL  EVENT-RELATED  INFCRMATICN 


•  Compute  Wtprwr  ekampuoana 

•  Oetennne  and  apply  ogaai  raters 

•  Estimate  covananca  and  'jma  oatay 


Flgwa  1.  ADIEEG-IV  system  for  quarfflfi  canon  of  event-related  brain  signals.  Setiaiote  subsystems  perform 
on-line  experimental  control  and  data  cotlecnon.  data  seiecson  ana  evaluation,  signal  processing  and  pat- 
tern  recognition.  Current  capacity  is  123  cn  ami  eta.  Sphertcai-nead  spatial  debiumng  modules  have  bees 
implemented,  and  mutnpie  source  modeling  aJgonthras  are  being  developed.  Digital  tapes  of  magnetic  reso¬ 
nance  images  or  of  elecrropnysiologieal  data  from  other  laboratories  are  convened  into  the  AD  I  EEC  data 
format  using  gateway  programs:  they  are  then  processed  using  me  same  program  modules  as  data  collected 


in  the  EEG  Systems  Laboratory,  f adapted  from  Cevins.  1987c; 
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CLASSIFIER-DIRECTED  artifact  detection 


Figure  —  Classifi er-dirceted  artifact  detection.  A  system  consisting  of  five  parallel  detectors  is  used  to  find 
contaminants.  Three  of  the  detectors  incorporate  layered-network  classifiers  to  choose  and  weigh  feature 
combinations.  In  tins  way,  a  precise  automated  procedure  replaces  sole  reliance  on  ad  hoc  waveform  detec¬ 
tors  and  manually  set  thresholds,  (adapted  from  Gevins  <4.  Morgan,  1 986) 


In  practice,  we  have  measured  times  erics 
covariances  (formerly  correlations)  between 
scalp  potentials  from  pain  of  channels,  either 
on  sets  of  single-trial  data  (Gevins  et  ah,  1981, 
1983,  1985),  or  on  enhanced  filtered  averages 
(Gevins  et  aL,  1987,  submitted  aj>,  in  prep). 
The  results  provide  interesting  new  informa¬ 
tion  about  brain  function,  and  suggest  that 
these  measures  are  worth  further  develop¬ 
ment.  The  scalp  patterns  we  have  measured 
have  been  consistent  with  the  known  func¬ 
tional  neuro anatomy  of  the  cerebral  cortex,  as 
determined  from  other  lines  of  evidence.  As 


predicted  by  neuro  anatomical  theory  and  clin¬ 
ical  neuropsychological  studies.  ERC  patterns 
corresponding  to  visual  stimulus  processing 
involved  posterior  sites  that  led  anterior  parie¬ 
tal  sites  and  premotor  sites,  and  ERC  patterns 
at  the  time  of  a  finger  pressure  involved  the 
midline  precentral  electrode  that  overties  the 
premotor  and  supplementary  motor  cortices. 
Although  the  problem  of  identifying -the  gen¬ 
erators  of  these  patterns  is  the  focus  of  current 
research,  it  has  not  yet  been  solved.  However, 
the  tendency  of  the  scalp  sites  involved  in 
these  patterns  to  be  spatially  separated,  wrih- 
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out  intervening  sites,  seems  to  rule  out  vol¬ 
ume-conducted  activity  from  just  one  or  two 
cortical  or  subcortical  generators. 

Procedures  for  Measurement 
of  Functional  Topography 

Currant  Cam put ar  Syatam 

The  current  computer  system  is  a  32-bit  mul¬ 
tiprocessor  system  with  3  computing  modes,  a 
12-MFLOP  floating  point  capability  and  a 
3500  megabyte  on-line  disk  capacity.  The  cur¬ 
rent  data  acquisition  system  is  capable  of  sam¬ 
pling  up  to  256  channels  at  up  to  2  kHz  sam¬ 
pling  rates  per  channel.  (Current  amplifica¬ 
tion  capabilities  are  123  channels.)  Trial  pres¬ 
entation.  which  is  performed  by  a  PC  con¬ 
trolled  by  the  host  computer,  is  automatically 
delayed  until  eye  blinks,  amplifier  setting 
from  eye  blinks,  and  gross  body  movements 
have  ail  died  down.  Up  to  70  channels  of 
EEG  and  non-EEG  channels  are  monitored  at 
a  time  on  a  color  graphics  screen  for  electrode 
problems.  Other  channels  are  bank-switchable 
to  the  monitor.  Averages  can  also  be  viewed 
on-line  to  check  for  event-registered  artifacts. 
Arafaos  such  as  eye  movement  and  muscle 
potential  contamination  are  automatically 
marked  by  multiple,  layered-network  pattern 
classification  programs  (Figure  2)  (Gevins  & 
Morgan.  1986). 

Spatial  Sampling 

The  use  of  64.  scalp  channels  provides  uniform 
scalp  coverage  with  an  intereiectrode  distance 
of  about  3-5  cm  on  a  typical  adult  head.  Fig¬ 
ure  3  shows  a  subject  wearing  a  stretchable 
64-channel  EEG  recording  cap.  The  elec¬ 
trodes  are  placed  on  the  cap  according  to  an 
expanded  version  of  the  10-20  system  (Figure 
4). 

We  are  currently  extending  EEG  record¬ 
ings  to  123  channels  'with  intereleorode  dis¬ 
tances  of  about  125  cm.  The  desirability  of  re¬ 
cording  from  so  many  sites  is  evident  since 
regional  cerebral  blood  flow  studies  suggest 
that  'conical  fields''  of  1—3  square  cenume- 


Fi gate  3-  Subject  wearing  64  channel  EEC  cap. 
(adapted  from  Gevins.  1988) 


e 


FI  jure  4.  Expanded  1 0-20  system  of  demode  posi¬ 
tion  nomenclature.  Additional  coronal  rows  of  elec¬ 
trodes  interpolated  between  the  International  10- 20 
System  coronal  rows  have  the  letter  “a*  for  'inte¬ 
rior”  added  to  the  designation  for  the  matt  roar  post¬ 
erior,  t-j.  a Pz  for  intenor  parietal  aadline  elec¬ 
trode.  With  64  electrodes,  the  average  distance  be- 
tween  electrodes  is  aoout  3.23  an.  (adapted  from 
Gervtns.  1987c) 
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ters  are  acavated  during  a  wide  variety  of  cog¬ 
nitive  tasks  (Roland,  1985a,b).  Yet,  while  the 
possibility  of  extracring  more  detailed  spatiaj 
information  has  been  dearly  demonstrated 
with  the  MEG  (see  Williamson  k  Kaufman, 
1987  for  discussion  and  bibliography),  it  is  not 
widely  appreciated  that  such  information  can 
be  obtained  from  appropriately  processed 
EEGs.  Examination  of  spatial  spectra  has 
shown  that,  if  information  is  available  from 
the  entirety  of  an  adult's  scalp,  adequate  sam¬ 
pling  would  require  more  than  128  electrodes 
at  2  cm  intervals  (Gevins,  1988). 

In  Figure  5,  we  can  see  the  etTect  of  an  in¬ 
creasing  number  of  E EG  electrodes  on  the 
electric  field  distribution  of  a  right  index  fin¬ 


ger  flexion.  Note  the  false  impression  of  local- 
izarion  'with  16  electrodes,  and  the  correct  ap¬ 
pearance  of  a  left  central  focus  with  27  and  51 
electrodes.  Although  there  is  much  current 
discussion  about  what  form  of  interpolation  is 
best  to  'use  in  making  colored  EEG  potential 
maps,  it  would  seem  that  the  more  compelling 
issue  is  the  need  for  more  eiemrodes. 

Rejecting  Artifact-Contaminated  EEG 
Signals 

An  interacrive  graphics  trial  editing  program 
is  used  by  an  operator  to  check  the  decisions 
of  the  automatic  artifact  detectors  (Gevins  k 
Morgan,  1986).  In  practice,  editors  have  be- 


Flfore  5.  Movement-locked  average  ERPs  at  78  ms  after  start  of  nght-index-iinger  flexion,  recorded  against 
a  linxed-ears  reference  with  (A)  16  channels.  (B)  27  channels.  (Cl  51  channels  and  (D)  after  aopiicanon  of  a 
Laolactan  operation  to  the  51  channel  ERPs.  There  is  a  false  localization  with  16  cnanneis  due  to  insuffi¬ 
cient  spatial  sampling.  The  27-  and  51-channel  ERP  recordings  show  the  true  potential  distribution  with  in¬ 
creasing  r-soluuon.  imorovement  in  topograohic  localization  with  the  Laoiacsan  is  self-evident.  iadaoted 
from  Gevins.  (988) 
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come  sufficiently  skilled  to  check  the  single 
trials  of  a  64-channel  recording  in  roughly 
twice  the  time  that  it  takes  to  run  the  experi¬ 
ment.  Algorithms  are  under  development  to 
recover  trials  with  non-saturating  artifacts. 
Eye  blinks  or  eye  movements,  for  example, 
can  be  removed  using  least  squares  noise  can¬ 
cellation,  given  EOG  reference  eiecrodes 
near  the  eyes. 

Finding  Trials  with  Olscsmibls  Evant-Ralatad 
Signals 

We  have  developed  a  simple  method  for  find¬ 
ing  trials  with  event-related  signals  which 
does  not  assume  that  event-reiated  signals  are 
discernible  in  every  trial,  and  which  has  mini, 
mai  assumptions  about  the  statistical  propert¬ 
ies  of  signal  and  noise  (Gevins,  1984;  Gevins 
et  al_  1986).  This  method  is  useful  for  increas¬ 
ing  the  signal-to-noise  ratio  of  average  ERPs, 
and  for  accentuating  the  differences  between 
averages  from  two  conditions  having  subtle 
cognitive  differences.  However,  it  is  not  an  es¬ 
sential  step  when  these  benefits  are  not  re¬ 
quired.  First,  a  mild  lowpass  filter  is  applied 
(3  dB  point  at  14  Hz  and  20  dB  down  at 
32  Hz)  and  the  data,  originally  sampled  at 
128  Hz,  are  decimated  by  a  factor  of  3  digitiz¬ 
ing  points.  Then,  a  “noise"  set  is  formed,  com¬ 
posed  of  segments  of  single  trials  for  each 
channel  that  are  randomly  timed  with  respect 
to  stimulus  and  response  events.  The  number 
of  points  in  each  segment  corresponds  to  that 
of  the  ERP  interval  to  be  investigated,  typi¬ 
cally  3  decimated  points  for  a  125  ms  interval. 

The  artificially-formed  “noise"  data  set  is 
then  compared  with  sets  of  single  trial  seg¬ 
ments  which  are  properly  time-registered  to 
stimulus  or  response  events.  This  comparison 
produces  equations  characterizing  the  event- 
reiated  signal  and  determines  a  list  of  trials 
with  detectable  event-reiated  signals.  A  pat¬ 
tern  classification  algorithm  construes  the 
equations  that  discriminate  between  sets  of 
single-trial  event-reiated  and  noise  segments 
for  each  channel.  The  equations  consist  of 
weighted  combinations  of  the  filtered  and  de¬ 
cimated  waveform  amplitude  values  within 
the  interval.  Three  leave-out-one-third-of-the- 
triais  validations  are  used,  and  the  average 
test-set  classification  is  compared  wuh  the  bi¬ 


nomial  distribution  for  significance.  If  the 
equations  are  significant  on  validation  data,  a 
consistent  event-reiated  signal  has  been  found 
in  many  of  the  trials.  ERPs  that  are  averaged 
over  these  selected  trials  are  called  “en¬ 
hanced**  averages,  because  the  signal  charac¬ 
teristics  have  been  accentuated  (Figure  6). 

Controlling  tor  Irralavant  Batwaan-Condltiott 
Variables 

In  comparin'  data  from  two  conditions,  it  is 
important  to  balance  the  two  data  sets  for 
stimulus-,  response-,  or  performance-related 
variables  that  have  differences  unrelated  to 
the  intended  comparison.  For  each  between- 
condition  comparison,  the  two  sets  of  event- 
related,  signal-bearing  trials  are  statistically 
balanced  far  each  subject  by  eliminating  trials 
that  have  outlying  values  for  these  irrelevant 
variables  (Gevins  et  al_  1981,  1983,  1985, 
1987,  submined  ajj,  in  prep). 

First,  a  program  automatically  calculates 
the  largest  subset  of  trials  that  behavioraily 
balance  the  two  conditions  being  compared, 
foe  a  requested  subset  of  behavioral  variables- 
The  set  of  about  50  behavioral  variables  in¬ 
dudes  response  time,  pressure,  velocity,  accel¬ 
eration,  duration  and  error,  as  well  as  stimulus 
parameters,  indices  of  muscle  activity  and 
“arousal"  (integrated  energy  in  the  Pz  elec¬ 
trode,  computed  in  ;00-ras  epochs  before  and 
after  the  onset  of  each  event).  This  is  followed 
by  an  interactive  program  that  displays  the 
means.  Student- s  t-tesis  and  histograms  of  the 
requested  variables  using  a  convenient,  win¬ 
dow-oriented  user  interface.  It  is  used  to 
check  the  automatic  split  of  the  distributions 
and  to  allow  their  correction  if  necessary.  Be¬ 
tween -condition  balance  is  achieved  when 
Student's  t  for  each  behavioral  variable  of 
comparison  has  a  significance  of  p  <  0.2. 

Radudng  Blur  Distortion  at  th»  Scalp 

Because  potentials  generated  by  sources  in  the 
brain  are  volume  conducted  through,  brain. 
cereorospinal  fluid,  skull  and  scalp  co  the  re¬ 
cording  electrodes,  potentials  from  a  localized 
source  are  spread  over  a  considerable  area  of 
scalp.  Potentials  measured  at  a  scalp  site  thus 
represent  the  summation  of  signals  from  many 
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Figure  6.  Use  of  pattern  recognition  analysis  to  re¬ 
move  trials  without  detectable  task-related  agnail 
from  a  set  of  single-trial  EB2H.  This  results  in  an  av¬ 
enge  EBP  with  a  higher  ngnal-to-ootse  ratio  ob¬ 
tained  from  fewer  triais.  (A)  An  onginal  avenge 
EBP  fanned  from  483  presentations  of  a  visual  nu- 
tnenc  stimulus.  (B)  Avenge  of  336  trials  with  con¬ 
sistent  event-related  signals  in  (he  P3  interval.  Trials 
were  selected  from  the  onginal  set  of  483  by  apply¬ 
ing  a  pattern  recognition  algorithm  to  distinguish  a 
320  ms.  P3  nmesenes  segment  from  a  pre-cue  ‘base¬ 
line'*  segment.  Note  the  greatly  increased  size  of  the 
event-related  peaks  and  lower  frequency  wave 
forma.  (C,  Avenge  of  147  tnais  which  did  not  have 
consistent  event-related  signals  in  the  i*3  interval. 
Note  the  relative  lack  of  event-related  acnvrty  in  the 
P3  interval,  (adapted  from  Gevtns  et  aU  1986) 


sources  over  much  of  the  brain.  (In  the  con¬ 
text  of  a  4-shell  spherical  head  model,  we 
have  estimated  the  amount  of  spread,  the 
‘point  spread,"  for  a  radial  equivalent  dipoie 
source  in  the  cortex  to  be  about  U  cm— Doyle 
&  Gevins,  1986).  This  spatial  low-pass  blur¬ 
ring  makes  source  localization  difficult,  even 
for  nearby  cortical  sources,  and  causes  the  po¬ 
tentials  from  local  sources  to  be  mixed  with 
those  from  more  distant  generators.  3y  model¬ 
ing  the  tissues  between  brain  and  scalp  as  sur¬ 
faces  with  different  thicknesses  and  resis¬ 
tances,  we  can  perform  a  debiurring  operation 
that,  in  principle,  makes  the  potential  appear 
as  it  wouid  if  it  were  recorded  just  above  the 
levei  of  the  brain  surface.  This  operation  can 
be  performed  without  imposing  assumptions 
as  to  the  actual  (cortical  or  subcorneal)  source 
locations.  The  debiurring,  however,  requires 
detailed  modeling  of  the  tissues,  which  is  a 
great  deal  of  work  when  the  exact  shape  of  the 
head  is  taken  into  account.  Furthermore,  there 
is  as  yet  no  good  solution  to  the  problem  of 
precisely  estimating  the  local  resistances  of 
skull  and  scalp.  When  the  EEG  scalp  electro¬ 
des  are  no  closer  together  than  about 
ZJ— 3.0  cm.  a  simpler,  model-free  debiurring 
method  provides  results  comparable  to  the 
more  complex  procedure. 

This  simpler  technique,  called  the  Lapla- 
cian  operator,  consists  of  computing  the  sec¬ 
ond  derivative  in  space  of  the  potential  field 
at  each  eiearode.  This  converts  the  potential 
into  a  quantity  proportional  to  the  current  en¬ 
tering  and  leaving  the  scalp  at  each  electrode 
site,  and  eliminates  the  erTec:  of  the  reference 
eleerode  used  during  recording.  An  approxi¬ 
mation  to  the  Laplaaan.  introduced  by  Hjorth 
(1975.  1980),  assumes  that  electrodes  axe  equi¬ 
distant  and  at  right  angles  to  each  other.  Al¬ 
though  this  approximation  is  fairly  good  for 
eleegodes  near  the  midline,  such  as  Cz.  it  be¬ 
comes  increasingly  poor  at  the  periphery.  An 
optimal  estimate  of  the  Lapiaaan  requires 
precise  knowledge  of  the  eieerode  positions 
in  three  dimensions,  information  which  can  be 
obtained  by  direct  measurements  from  the 
head  (Greer  &  Gevins,  in  prep).  A  property 
computed  Laolaaan  operator  can  produce  a 
dramatic  improvement  in  the  spanai  topogra¬ 
phic  detail  of  ERP  comoonents  see  Figure 
5D>. 
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Choosing  Filters  and  Analysis  Windows  with 
Wlgnar  Time-frequency  Distributions 

The  ERP  waveform  is  a  function  of  time  and 
does  not  provide  explicit  frequency  informa¬ 
tion.  Yet.  the  instantaneous  frequency  is  not 
constant  for  different  ERP  components.  For 
each  component,  the  instantaneous  frequency 
must  be  determined  to  isolate  that  component 
with  digital  filtering,  and  the  duration  over 
which  that  frequency  is  stable  must  be  deter¬ 
mined  to  define  the  length  of  the  analysis  in¬ 
terval.  Power  spectra  of  ERP  waveforms  pro¬ 
vide  frequency  information  but  obscure  time- 
dependent  phenomena.  A  view  of  the  spec¬ 
trum  as  it  changed  over  time  would  give  a  new 
view  of  the  evolution  of  different  frequency 
components  of  the  ERP.  A  simple  but  ineffec¬ 
tive  approach  would  be  to  compute  the  spec¬ 
trum  over  highly  overlapped  windows  of  the 
average  ERP.  A  preferable  metnod  is  to  com¬ 
pute  a  general  function  of  time  and  frequency, 
called  the  Wigner  Distribution,  which  approx¬ 
imates  the  instantaneous  energy  for  a  given 
time  and  frequency.  In  practice,  enhanced 
ERPs  show  strong  enough  energy  “peaks"  in 
the  Wigner  Distribution  that  very  simple  inter¬ 
pretations  of  the  time  and  frequency  locations 
of  signal  energy  are  valid  (Morgan  4  Gevins. 
1986;  Figure  7).  From  visual  inspection  of 
Wigner  Distributions,  it  is  a  simple  matter  to 
specify  digital  filter  characteristics  that  pro¬ 
duce  optimal  time-frequency  resolution  for  a 
given  ERP  component,  and  to  determine  the 
frequency-stable  interval  for  that  component 
This  procedure  need  not  be  implemented  for 
each  experiment  For  example,  a  filter  and  in¬ 
terval  defined  for  the  N 100  in  one  expenment 
may  be  sufficient  for  analyzing  the  N100  in 
another  expenment 

Computing  Event-Related  Covariances 
(ERCs)  Between  CTtsnneis 

To  summarize,  these  preliminary  steps  are  fol¬ 
lowed  before  computing  ERCs: 

1.  Record  a  sufficient  amount  of  data  using  as 
many  electrodes  as  possible. 

2.  Remove  data  wuh  artifact  contamination. 

3.  Find  tnais  wuh  consistent  event-related  sig¬ 
nals  (optional). 


A  Select  the  pair  of  conditions  to  be  com¬ 
pared.  and  eliminate  tnais  with  extreme 
values  of  behavioral  variables  to  obtain  be- 
havioraily  balanced  trial  sets. 

5.  Apply  the  Lapladan  operator  to  the  poten¬ 
tial  distribution  of  each  non-peripheral 
scalp  electrode  location. 

6.  Compute  Wigner  Distribution  on  average 
ERP  (optional)  and  determine  digital  filter  . 
characteristics  and  analysis  intervals. 

Once  the  data  sets  have  been  prepared  in 
this  way,  the  next  step  is  to  compute  the  en¬ 
hanced,  filtered  and  decimated,  averaged  La- 
placian  ERP  for  each  condition.  Then,  multi¬ 
lag  crosscovariance  functions  are  computed 
between  ail  pairwise  channel  combinations  of 
these  averaged  ERPs  in  each  selected  analysis 
window.  The  magnitude  of  the  maximum 
value  of  the  crosscovariance  function  and  its 
lag  time  are  the  features  'used  to  characterize 
the  ERC.  The  covariance  analysis  interval  is 
the  width  of  one  period  of  the  band-center  fre¬ 
quency  of  each  filter.  Down-sampling  factors 
are  determined  by  the  20  dB  rejection  point, 
and  the  covariance  function  is  computed  up  to 
a  lag-time  of  one-haif  period  of  the  high  fre¬ 
quency  for  each  band.  For  example,  we  often 
use  a  filter  with  3  dB  cutoffs  at  -i  and  7  Hz. 
and  with  20  dB  attenuation  it  1_5  and  9_5  Hz. 
The  filtered  amesenes  are  decimated  from 
123  Hz  to  21  Hz  for  each  covariance  calcula¬ 
tion.  Covariance  is  estimated  over  a  187-ms 
window,  which  corresponds  to  one  period  of  a 
f-ti  Hz  sinusoid.  Each  window  is  lagged  by  up 
to  3  lags  at  the  original  undecimated  sampling 
rate,  i.e_  1/123  s  per  lag. 

Estimating  the  significance  of  ERCs  re¬ 
quires  an  estimate  of  the  standard  deviation  of 
the  "noise"  ERC  First,  random  intervals  in 
each  single  trial  of  the  ensemble  are  averaged. 
Then.  ERC  analysis  is  performed  on  a  filtered 
and  decimated  version  of  the  resulting  "noise" 
averages,  yielding  a  distribution  of  “noise" 
ERCs.  3ecause  of  the  large  number  of  chan¬ 
nel  pairs,  some  spurious  significant  covar¬ 
iances  may  be  found.  Therefore,  hie  threshold 
for  significance  is  reduced  according  to  the  di¬ 
mensionality  of  the  data  with  Duncan's  cor¬ 
rection  procedure.  The  number  of  channels  is 
used  as  a  conservative  estimate  of  the  number 
of  independent  dimensions.  The  most  signifi- 
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(ff)  The  pseudo- Wigner  distribution  of  the  analytic  signal  of  the  same  data.  This  representation  shows  that 
the  event-related  processes  are  changing  rapidly  in  both  time  and  frequency.  The  first  moment  along  the 
time  axis  for  each  frequency  is  the  group  delay,  while  the  first  moment  along  the  frequency  axis  is  the  in¬ 
stantaneous  frequency.  There  is  a  buildup  in  energy  after  the  stimulus,  and  a  general  increase  in  freonency 
until  the  energy  concentration  between  the  time  of  the  N 1  and  71  peaks  begins  to  fail  oiT  (most  prominent 
in  the  Pz  channel).  Then  there  is  a  glide  down  in  frequency  in  the  no-move  task  (most  prominent  in  the  Pi 
and  aOz  channels i.  which  culminates  in  a  concentration  of  energy  around  the  time  of  the  P3  peak,  (adapted 
from  Gevms  I98<*\ 
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cant  ERCs  in  each  interval  are  graphed  (Fig- 
ore  8). 

To  compare  ERC  maps  between  condi¬ 
tions,  the  differences  in  means  of  significant 
ERCs  between  conditions  are  tested  with  an 
ANOVA  and  post-hoc  t-tests.  The  similarity 
between  two  multivariate  ERC  maps  is  mea¬ 
sured  with  an  estimate  of  the  correlation  be¬ 
tween  them.  The  estimate  comes  from  a  distri¬ 
bution-independent  -bootstrap"  Monte  Carlo 
procedure  (Efron,  19701,  which  generates  an 
ensemble  of  correlation  values  from  randomly 
selected  choices  of  the  repeated  measures. 
This  also  yields  a  confidence  interval  for  the 
estimates. 

Vmtldatlng  Slgntflcmnca  of  ERCs 

We  test  the  between-subjecr  variability  of 
ERC  patterns  by  determining  whether  each 
pair  of  experimental  conditions  of  a  particular 
subject  can  be  distinguished  using  discrimi¬ 
nating  equations  generated  on  the  other  sub¬ 
jects.  Likewise,  we  determine  wiihin-subjea 
reliability  by  attempting  to  discriminate  the 
experimental  conditions  for  each  session  us¬ 
ing  equations  generated  on  that  subject’s 
other  sessions.  These  tests  are  performed  on 
sets  of  single  trials  to  quantify  the  extent  to 
which  the  condition-specific  patterns  from  the 
ERC  analysis  of  the  average  ERPs  are  observ¬ 
able  in  each  tnaL  Although  this  procedure 
could  be  done  with  any  type  of  discriminant 
analysis,  we  have  developed  the  use  of  distri¬ 
bution-independent,  “neural  network"  pattern 
classification  algorithms  for  this  purpose  ;Ge- 
vins  et  al„  1979aJj.c,  1981,  1983,  1985,  1986, 
1987;  Gevins  Morgan.  1986;  Gevins,  1980, 
1984, 1987b).  We  have  shown  that  this  method 
has  bener  sensitivity  and  specificity  than  step¬ 
wise  or  full-model  linear  or  quadratic  discri¬ 
minant  analysis  (Gevins,  1980).  The  pattern 
recognition  approach  has  the  advantage  of 
testing  how  weil  a  subject's  individual  trials 
conform  to  those  of  the  group  in  discriminat¬ 
ing  two  behavioral  conditions  of  interest.  In 
the  same  way,  the  trials  of  each  session  of  a 
subject  are  tested  by  conformity  to  trials  from 
the  other  sessions  of  that  subject.  Requiring 
tnai-by-tnai  discriminabiiity  is  a  stria  condi¬ 
tion  for  heading  berween-subjea  variability 
and  within-subjem  reliability. 


Figure  8.  Schematic  diagram  showing  die  relation¬ 
ship  of  an  event-related  covariance  (ERC,  line  on  a 
top-view  of  a  model  head  (left)  to  the  theta-band-fil- 
tered.  averaged  evrnr-reiated  Lapiaaan  derivation 
waveforms  (right).  ERCs  were  computed  over  the 
indicated  187-ms  analysis  interval  from  the  apt  and 
aCc  electrode  sites.  The  width  of  an  ERC  line  indi¬ 
cates  the  significance  of  the  covariance  between  two 
waveforms,  with  the  scale  appearing  above  the  word 
“significance.*  The  color  of  the  line  indicates  the 
time  delay  in  ms  (lag  time  of  maximum  covariance) 
as  shown  in  the  scale  above  *ms  delay.*  The  color 
of  the  arrow  indicates  the  sign  of  the  covariance 
(same  color  as  line  -  positive;  skin  color  -  nega¬ 
tive).  The  arrow  points  from  leading  to  lagging 
channel,  unless  there  is  no  delay,  in  which  case  a 
bar  is  shown.  The  covariance  berween  aPr  and  aCz 
is  significant  at  p  <  lO^.The  apx  waveform  leads 
the  aC:  waveform  by  about  16—31  ms  (green  line), 
and  the  covariance  is  positive  (arrow  also  green), 
(from  Gevins  et  ac,  submitted  a) 

Each  subject's  classification  yields  a  score, 
which  is  the  percent  of  trials  that  are  correctly 
classified  by  the  group  discrimination  equa¬ 
tions.  The  score  is  assessed  for  sigmiicancc  by 
comparison  to  the  binomial  distribution  (Ge¬ 
vins,  1980).  A  significant  classification  score 
for  a  subject  indicates  that  the  group  equa¬ 
tions  are  successful  in  discriminating  the  two 
conditions  in  his  or  her  trials. 

Within-subjecx  (between-session)  reliability 
is  tested  in  a  similar  manner.  The  trial  set 
(consisnng  of  the  two  conditions)  from  each 
of  a  suojea's  sessions  is  tested  with  equations 
developed  on  the  trial  sets  from  his  or  her 
other  sessions.  The  single-tnai  ERC  values 
come  from  channel  pairs  that  are  significant 
in  the  ERC  partem  formed  from  the  average 
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over  ail  his  or  her  sessions.  Post-hoc  comparis¬ 
ons  are  valuable  in  determining  whether  ef¬ 
fects  of  learning  and/or  habituation  are  evi¬ 
dent  over  sessions,  by  indicating  which  ses¬ 
sions  are  alike,  and  where  transitions  occur 
between  sessions. 


Application  of  EHC  Analysis 

Study  of  VJauomotor  Partormmnca 

Procedure  (Gevins  et  ai..  1987,  submitted  a,b). 

Seven  heaithy,  right-handed  male  adults  par¬ 
ticipated  in  this  study.  A  visual  cue,  slanted  to 
the  right  or  to  the  left,  indicated  to  subjects  to 
prepare  to  make  a  response  pressure  with  the 
right  or  left  index  finger.  One  second  later,  the 
cue  was  followed  by  a  visual  numeric  stimulus 
(number  I— 9)  indicating  that  a  pressure  of  0.1 
to  0.9  kg  should  be  made  with  the  index  finger 
of  the  hand  indicated  by  the  cue.  Feedback  in¬ 
dicating  the  exact  response  pressure  produced 
was  presented  as  a  two-digit  number  one  sec¬ 
ond  after  the  peak  of  the  response  pressure. 
On  a  random  20%  of  the  trials,  the  stimulus 
number  was  slanted  opposite  to  that  of  the 
cue,  and  subject  were  to  withhold  their  re¬ 
sponses  on  these  ‘catch  trials.”  The  next  trial 
fallowed  1  s  after  disappearance  of  the  feed¬ 
back.  Subjects  each  performed  several  hun¬ 
dred  trials,  'with  rest  breaks  as  needed. 

Twenty-six  channels  of  EEG  data,  as  weil 
as  vertical  and  horizontal  eye-movements  and 
flexor  digitori  muscle  activity  from  both  arms, 
were  recorded.  Ail  singie-triai  EEG  data  were 
screened  for  eye  movement,  muscle  potential 
and  other  artifacts.  Contaminated  data  were 
discarded. 

Intervals  used  for  ERC  analysis  were  cen¬ 
tered  on  major  ERP  peaks.  ERCs  were  com¬ 
puted  between  averaged  Laplarian  ERPs  from 
each  of  the  120  pairwise  combinations  of  the 
16  nonoenpherai  channels  in  intervals  from 
500  ms  before  cue  to  500  ms  after  the  feed¬ 
back. 

Data  sets  were  separated  into  trials  in 
which  subsequent  performance  was  either  ac¬ 
curate  or  inaccurate.  Accurate  and  inaccurate 
performance  trials  were  those  in  which  the  er¬ 


ror  (deviation  from  required  finger  pressure) 
was  less  than  or  greater  than,  respecnvdy,  the 
mean  error  over  the  recording  session. 

Results  and  Discussion 

The  ERC  pattern  for  the  interval  at  the  peak 
of  the  linger  pressure  (Figure  9)  closely  corre¬ 
sponded  to  prior  functional  neuroanatomicai 
knowledge,  lending  a  first  level  of  validation 
for  the  patterns  associated  with  higher-order 
cognitive  acriviry.  The  midline  precentral  elec¬ 
trode,  overlying  the  premotor  and  supplemen¬ 
tary  motor  cortex,  was  the  focus  of  ail  move¬ 
ment-related  patterns,  and  the  patterns  for  the 
two  hands  were  appropriately  Iateraiized, 
clearly  reflecting  the  sharply  focused  current 
sources  and  sinks  spanning  the  hand  areas  of 
motor  cortex. 

ERC  patterns  during  a  375-ms  interval  cen¬ 
tered  687-ms  post-cue  (spanning  the  late  Con¬ 
tingent  Negative  Variation:  CNV)  were  dis¬ 
tinct  from  those  related  to  overt  finger  re¬ 
sponses.  The  pattern  associated  with  subse¬ 
quently  accurate  right-hand  performance  in¬ 
volved  predominantly  left  hemisphere  sites, 
particularly  left  frontal  and  appropriately  la- 


Figure  9.  Most  significant  (too  standard  deviation) 
ERC  patterns  at  the  peak  of  the  finger  pressure  for 
right  and  left  hand  tnais  from  seven  people,  super¬ 
imposed  on  maos  of  Laolaaan  response  potential 
amoiitude.  Note  that  the  anterior  midline  precentral 
(aCz)  electrode  is  the  focus  of  all  covariances,  with 
16—31  ms  time  delays  between  aCz  and  Fz.  The  pat¬ 
terns  are  distinctly  iateraiized  according  to  respond¬ 
ing  hand.  The  sign  of  the  aCz  covariances  is  pasiuve 
for  lateral  frontal,  and  negauve  for  lateral  central 
and  anterior  parietal  electrodes. 
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teralized  central  and  parietal  sites  (Figure 
10A).  The  panem  preceding  accurate  left- 
hand  performance  involved  predominantly 
right  hemisphere  sites,  in  addition  to  the  left 
frontal  site.  Inaccurate  performance  by  the 
right  hand  (Figure  10B)  was  preceded  by  a 
highly  simplified  pattern,  while  inaccurate 
performance  by  the  left  hand  was  preceded  by 
a  complex,  spatially  diffuse  pattern. 

When  the  trials  of  each  of  the  7  subjects 
were  classified  by  equations  developed  on  the 
trials  of  the  other  6  subjects,  the  overall  dis¬ 
crimination  was  59%  <,p  <  0.01)  for  right  band 
and  57%  (p  <  0.01)  for  left-hand  perform¬ 
ance.  For  the  subject  with  the  most  trials,  aver¬ 
age  classification  of  63%  (p  <  .001)  for  subse¬ 
quent  right-  and  62%  (p  <  .01)  for  subsequent 
left-hand  performance  was  achieved  by  test¬ 
ing  a  separate  equation  on  each  fifth  of  his  tri¬ 
als.  formed  from  the  other  four-fifths. 

We  suggest  that  our  pre-stimulus  ERC  pat¬ 
terns  characterize  a  distributed  preparatory 
neural  set  related  to  the  accuracy  of  subse¬ 
quent  task  performance.  This  set  appears  to 
involve  distinctive  cognitive  (frontal),  integra¬ 
tive-motor  and  lateralized  somesiheric-motor 
components.  The  involvement  of  the  left-fron¬ 
tal  site  is  consistent  with  clinical  findings  that 


preparatory  sets  are  synthesized  and  integra¬ 
ted  in  prefrontal  cortical  areas,  and  with  ex¬ 
perimental  and  clinical  evidence  indicating  in¬ 
volvement  of  the  left  dorsolateral  prefrontal 
cortex  in  delayed  response  tasks.  A  midline 
precentral  integrative  motor  component  is 
consistent  with  known  involvement  of  premo¬ 
tor  and  supplementary  motor  areas  in  initiat¬ 
ing  motor  responses.  The  finding  of  appropri¬ 
ately  lateralized  central  and  parietal  compo¬ 
nents  is  consistent  ’with  evidence  from  pri¬ 
mates  and  humans  for  neuronal  firing  in  mo¬ 
tor  and  somatosensory  cortices  prior  to  motor 
responses. 

Study  at  Incipient  Fatigue 

Procedure  (Gevins  et  ai_  in  prep) 

After  learning  and  practicing  a  battery  of 
tasks  until  their  performance  was  stable  on 
one  day,  each  of  five  right-handed,  healthy 
male  subjects  returned  to  the  laboratory  the 
next  morning  and  performed  the  tasks  for 
about  6  hours.  Following  a  dinner  break,  they 
resumed  task  performance  for  an  additional  6 
to  3  hours. 


Figure  10.  View  of  the  significant  (p  <  0.05)  Contingent  Negative  Vanadon  (CNV)  ERC  patterns,  superim¬ 
posed  on  a  mid-sagmal  .Magnetic  Resonance  Image  of  a  subject's  cranium.  Measurements  are  from  an  inter¬ 
val  500  to  375  ms  after  the  cue  jr  subsequently  accurate  (A)  and  inaccurate  f  B)  right-hand  visuomotor  task 
performance  by  seven  nght-nanded  men.  The  thidenes*  of  an  ERC  line  is  proooruonai  to  its  significance 
(from  .05  to  .QOS).  A  yellow  line  indicates  the  ERC  is  positive,  while  a  red  line  is  negative.  Covariances  in¬ 
volving  left  frontal  fF3),  left  central  (C3),  let!  parietal  (P3%  and  left  amero-panetai  (aPl)  electrode  sites,  ail 
contralateral  to  the  subsequently  responding  hand,  are  the  most  prominent  sites  in  the  panem  for  subse¬ 
quently  accurate  performance.  Only  two  covariances  characterize  suosequemly  inaccurate  performance:  let) 
frontal  (FE)  with  left  oanetai  (P3),  and  left  frontal  with  left  amero-panetai  (apll  sites. 
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There  were  four  tasks  in  the  battery,  includ¬ 
ing  easy  and  difficult  continuous  and  discrete 
visuomotor  tracking  tasks,  a  simple  numeric 
memory  task,  and  a  difficult  visuomotor  mem¬ 
ory  task  (VMMT).  Since  we  expected  that 
eariy  neurai  signs  of  fatigue  would  be  most 
evident  during  demanding  tasks,  we  analyzed 
the  VMMT  first.  This  task  required  subjects  to 
remember  two  continuously  changing  num¬ 
bers,  in  the  presence  of  numeric  distraaors,  in 
order  to  produce  precise  finger  pressures. 
Each  trial  consisted  of  a  warning  symbol  fol¬ 
lowed  by  a  single-digit  visual  stimulus  to  be 
remembered,  followed  by  the  subject's  linger- 
pressure  response  to  the  stimulus  number  pre¬ 
sented  two  trials  earlier,  followed  by  a  2-digit 
feedback  number  indicating  the  accuracy  of 
the  response.  For  example,  if  the  stimulus 
numbers  in  tlve  successive  trials  were  8,  6,  1, 
9,  -t,  the  correct  response  would  be  a  pressure 
of  0.3  kg  when  seeing  the  I,  0.6  kg  for  the  9, 
and  0.1  kg  for  the  4.  To  increase  the  task  diffi¬ 
culty,  subjects  were  required  to  withhold  their 
response  on  a  random  20%  of  the  trials.  These 
“no-response  catch  trials”  were  trials  in  which 
the  current  stimulus  number  was  identical  to 
the  stimulus  two  trials  earlier. 

Trials  eariy  in  the  recording  session  with 
accurate  finger  pressures  formed  the  “Alert" 
data  set.  Trials  from  eariy  in  the  evening, 
when  performance  was  just  starting  to  decline, 
formed  the  “Incipient  Fatigue"  data  set.  For 
each  subject,  trials  with  relatively  inaccurate 
responses  were  then  deleted  from  the  Incipi¬ 
ent  Fatigue  data  set  so  that  the  final  Alert  and 
Incipient  Fatigue  data  sets  consisted  of  trials 
with  equivalently  accurate  performance.  This 
crucial  step  allowed  measurement  of  neuro- 
eiectnc  patterns  associated  with  incipient  fa¬ 
tigue  whiie  controlling  for  those  due  to  vari¬ 
ations  in  performance  accuracy. 

EEGs  were  recorded  with  either  33  or  51 
channels  with  a  nylon  mesh  cap.  Vertical  and 
horizontal  eye  movements  were  also  recorded, 
as  were  the  responding  flexor  digitori  musde 
potentials.  electrocardiogram  and  respiration. 
Three-axis  Magnetic  Resonance  Image  scans 
were  made  of  3  of  the  5  subjeea. 

Grand-average  (over  the  five  pilots)  ERPs 
were  time-locked  to  presentadon  of  the  nu¬ 
meric  stimulus.  Incident- Fangue  ERPs  were 
suotracted  from  Alert  ERPs  in  order  to  hign- 


ligfat  changes  due  to  fatigue.  Spano temporal 
neuroeiectric  patterns  were  then  quantified  by 
measuring  SRCs  between  ail  153  pairwise 
combinations  of  the  13  nonperipheral  eiecro- 
des.  ERCs  were  measured  across  brief  seg¬ 
ments  of  grand-average  AJert-minus-tndpi- 
ent-Farigue  subtraction  ERPs.  The  first  ERC 
interval  was  500  ms  wide  and  was  centered 
312  ms  before  the  numeric  stimulus.  The  next 
two  ERC  intervals  were  1 87  ms  wide  and  were 
positioned  'with  respect  to  the  N125  peak  ei- 
idted  by  the  numeric  stimulus,  and  the  P380 
ERP  peak  eiidted  by  the  infrequent,  no-re¬ 
sponse  catch-trial  stimuli. 

Results  and  Discussion 

A  number  of  significant  Alert-minus-Indpi- 
ent- Fatigue  ERCs  were  found  during  the 
500-ms  prestimulus  interval.  Significance  was 
determined  by  comparison  to  a  distribution  of 
“noise"  ERCs,  and  corrected  with  Duncan's 
procedure,  following  the  description  above 
(“Computing  Event- Related  Covariances  Be¬ 
tween  Channels”).  Midline  central,  left  parie¬ 
tal.  left  intern  parietal,  right  antero  parietal 
and  right  posterior  parietal  electrodes  were 
the  major  ERC  fod.  There  were  no  significant 
ERCs  in  the  interval  centered  at  62  ms  post- 
sum  ulus.  The  ERCs  computed  over  the  P380 
no-response  difference  ERP  were  focused  on 
the  nridline  precentral,  and  right  anterior  and 
posterior  parietal  electrodes. 

The  ERC  changes  with  Indpient  Fatigue 
suggest  that  dynamic  funcrional  neural  net¬ 
works  associated  with  specific  cognitive  func¬ 
tions  are  selectively  affeced  during  early  fa¬ 
tigue.  During  the  presdmuius  interval,  when 
subjects  were  maintaining  the  last  two  visually 
presented  numbers  in  working  memory  and 
preparing  for  the  next  stun  ulus.  ERCs  de¬ 
creased  in  number  in  the  Indpient  Fatigue 
condition.  The  lack  of  ERC  differences  be¬ 
tween  Alert  and  Indpient  Fatigue  condidons 
during  the  interval  centered  at  62  ms  suggests 
that  the  “exogenous"  stages  of  visual  stimulus 
processing  are  reiadveiy  unaffected  by  eariy 
fatigue.  However,  during  the  later  post-stimu¬ 
lus  interval  of  tnais  requiring  an  inhibidon  of 
the  response.  ERCs  again  decreased  in  num¬ 
ber  wuh  Incipient  Faugue. 
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The  results  suggest  that  although  neural 
systems  responsible  for  primary  visual  stimu¬ 
lus  processing  are  relatively  unaffected  by  In¬ 
cipient  Fatigue,  cortical  associative  areas  re¬ 
sponsible  for  higher  cognitive  functions  such 
as  working  memory  rehearsal,  preparation, 
and  motor  inhibition  are  altered  prior  to  ap¬ 
preciable  degradations  in  performance. 


Conclusions 

The  idea  of  distributed  processing  networks  in 
the  brain  has  been  weil  established  in  die  lit¬ 
erature.  dating  back  to  Sherrington.  This  pow¬ 
erful  concept  has  been  under-utilized  in  neu- 
rocogmtive  studies,  perhaps  because  of  the 
difficulty  of  measuring  activity  patterns.  Re¬ 
cent  attempts  at  constructing  ■‘parallel  distrib¬ 
uted  processing"  models  of  cognition  (Rumei- 
hart  et  al_  1986)  aim  to  derive  computational 
principles  from  knowledge  of  the  brain’s  own 
distributed  processing  networks.  These  efforts 
are  encouraging,  yet  they  depend  on  contin¬ 
ued  characterization  of  those  networks.  Mea¬ 
surement  of  the  dynamic  functional  topogra¬ 
phy  of  the  brain,  the  time-varying  statistical 
relations  among  the  field  potentials  from  a 
distributed  set  of  recording  sites,  is  an  empiri¬ 
cal  means  of  accomplishing  this.  It  has  been 
shown  that  patterns  of  event-related  covar¬ 
iance.  measuring  functional  topography,  are 
related  to  cognitive  processing,  and  thus  may 
represent  active  states  of  a  distributed  process¬ 
ing  network. 

At  least  three  major  factors  have  contrib¬ 
uted  to  our  ability  to  begin  characterizing  the 
functional  topography  of  the  human  brain. 
First  was  the  use  of  sufficiently  controlled 
cognitive  tasks  to  allow  inference  of  the  cogni¬ 
tive  processes  taking  place  during  brief  analy¬ 
sis  intervals.  Along  with  this  was  the  produc¬ 
tion  of  computer  programs  to  choose  trials 
from  two  experimental  conditions  that  are  bal¬ 
anced  for  important  behavioral  variables.  This 
factor  created  the  functional  context  for  topo¬ 
graphic  measurements.  Second  '-“'as  the  ability 
to  record  from  a  sufficient  number  of  chan¬ 
nels  to  adequately  sample  the  available  re¬ 


cording  surface  of  the  head.  Indications  are 
that  even  with  64  evenly  spaced  scalp  electro¬ 
des,  there  is  potentially  more  information  to 
be  gained  from  greater  spatiai  sampling.  This 
factor  allowed  the  topography  to  be  repre¬ 
sented  spatially.  Third  was  the  application  of 
sophisticated  signal  processing  techniques  to 
the  analysis  of  event-reiated  potentials.  This 
factor  was  necessary  to  appropriately  measure 
die  inter-regional  relations  comprising  the 
functional  topography. 

Neurocognitive  Pattern  (NC?)  Analysis 
and  Event-Related  Covariance  (ERC,  Analy¬ 
sis  should  be  distinguished  from  brain  eiecm- 
cai  activity  maps,  which  are  usually  color 
topographic  displays  interpolated  from  16—20 
channels  of  averaged  ERPs  or  EEG  specau. 
or  the  difference  between  such  measures  and 
a  set  of  normative  data.  We  use  more  exten¬ 
sive  signal  processing  and  panem  recognition 
algorithms  to  reduce  volume  conduction  ef- 
feca  and  to  extract  event-related  signals  from 
unrelated  background  noise  of  the  brain,  com¬ 
pute  between -channel  ERC  patterns,  and  dis¬ 
play  their  scalp  distribution  in  3-D  perspective 
graphics  of  the  head  and  brain.  Subtle  aspects 
of  neurocognitive  function,  such  as  the  mea¬ 
surement  of  preparatory  sets  that  precede  ac¬ 
curate  performance,  are  revealed  by  these 
ERC  patterns  but  are  not  necessarily  apparent 
on  topographic  maps  (Gevins  et  al_  submitted 
b).  ERC  .Analysis  is  currently  undergoing  fur¬ 
ther  development.  The  value  of  scalp  patterns 
could  be  greatly  expanded  by  a  more  ade¬ 
quate  understanding  of  their  sources.  The  dif¬ 
ficult  problem  of  determining  the  distributed 
source  network  in  the  brain  producng  the 
scalp  ERC  patterns  is  cruaai.  and  is  the  focus 
of  our  ongoing  work. 

It  is  a  testimony  to  the  ingenuity  of  cogni¬ 
tive  psychologists,  psychopbysioiogists,  neu¬ 
rologists  and  psychiatrists  that  so  much  has 
been  learned  about  the  timing  of  neurocogni¬ 
tive  processes  using  very  modest  recording 
equipment  and  analysis  techniques.  It  is. 
therefore,  almost  certain  that  when  more  ad¬ 
vanced  recording  methods  and  more  powerful 
analytic  tools  are  widely  available,  raoid  ad¬ 
vances  in  understanding  the  neural  basis  of 
human  cognitive  functions  will  take  place. 
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Summary  A  new  method  that  measures  between-channeL  event-related  covariances  t  ER Cs>  from  scalp-recorded  brain  signals 

has  been  developed.  The  method  was  applied  to  recordings  of  36  EEG  channels  from  5  right-handed  men  performing  a  bimanual 
visuomotor  judgment  task  that  required  fine  motor  controL  Covariance  and  time-delay  measures  were  denved  from  pairs  of  filtered, 
laplacian-dertved.  averaged  wave  forms,  which  were  enhanced  by  retention  of  outlying  trials,  in  intervals  spanning  event-related 
potential  components.  Sumulus-  and  response-locked  ERC  patterns  were  consistent  with  funcuonai  neuroanatomicai  models  of 
visual  stimulus  processing  and  response  execution.  In  early  post-stimulus  intervals.  ERC  patterns  differed  according  to  the  physical 
properties  of  the  stimulus:  in  later  intervals,  the  patterns  differed  according  to  the  subjective  interpretauon  of  the  sumulus.  The 
response-locked  ERC  patterns  suggested  4  major  conical  generators  for  the  voluntary  fine  motor  control  required  by  the  task:  motor, 
somesthejic.  premotor  and/or  supplementary  motor,  and  .prefrontal.  This  new  method  may  thus  be  an  advancement  toward 
characterizing,  both  spauaily  and  temporally,  funcuonai  cortical  networks  in  the  human  brain  responsible  for  percepuon  and  acuon. 

Key  words:  Event-related  covariances:  Event-related  potentials;  Lapladan  denvauon:  Spatio-temporal  maps:  N100;  P300;  Response 
poienuals;  Functional  conical  networks 


This  study  was  undertaken  to  test  the  hypothe¬ 
sis  that  human  task  performance  involves  coordi¬ 
nated  processing  of  information  by  different  areas 
of  the  brain.  Prior  studies  have  shown  that  when  a 
brain  region  becomes  involved  in  a  task,  some 
subset  of  neurons  in  that  region  becomes  synchro¬ 
nized.  and  the  synchronization  is  manifested  as  a 
change  in  the  wave  shape  of  its  extraceilulariy 


1  A  part  of  the  results  concerning  the  covariance  pattern  of  the 
response  event  reported  here,  has  been  summarized  in  Gevins 
et  al.  1987.  This  paper  presents  a  detailed  descnpuon  of  the 
complete  experimental  design,  the  methods  of  analysis.  ERR 
and  event-related  covariance  results  of  'exogenous'  and  'en¬ 
dogenous'  sumulus-locked  data  and  response-locked  data,  and 
a  discussion  relating  these  results  to  other  psychophvsiological 
research. 

Correspondence  to:  A.S.  Gevins.  EEG  Systems  Laboratory, 
51  Federal  Street,  San  Francisco,  CA  94107  (U.&A.). 


recorded  macropotential  (John  1967;  HuJ  1972; 
Freeman  and  Skarda  1985;  Freeman  1987).  Low- 
frequency  components,  in  particular,  have  been 
shown  to  be  important  for  visuomotor  and  cogni¬ 
tive  tasks  (Legewie  e:  al.  1969;  Ishihara  and  Yoshi 
1972;  Doyle  et  al.  1974;  Komisaruk  1977;  Gevins 
et  al.  1979a.b,c).  We  hypothesize,  therefore,  that 
during  performance  of  such  tasks,  coordinated 
processing  will  be  reflected  m  the  similarity  of 
low-frequency  potential  wave  shape  among  the 
regions  involved.  Since  wave  shape  similarity  of 
macropotentials  from  different  areas  of  the  brain 
can  be  measured  by  covariance  and  correlation 
(reviewed  in  Gevins  1987;  Gevins  and  Bressler 
1988),  these  techniques  can  characterize  the  spa¬ 
tial  organization  of  coordinated  low-frequency  ac¬ 
tivity. 

Both  animal  and  human  studies  provide  further 
evidence  for  this  reasoning.  Animal  studies  using 
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behavioral  conditioning,  for  example,  have  shown 
that  the  functional  coordination  of  neuronal  popu¬ 
lations  is  reflected  in  the  synchronization  of  their 
macropotentials  (possibly  with  time  delay)  specifi¬ 
cally  in  the  theta  (4-7  Hz)  frequency  band  (Adey 
et  al.  1961;  Efremova  and  Trush  1973;  Livanov 
1977).  Dumenko  (1970)  found  that  the  correlation 
between  extracellular  potentials  of  visual  and  mo¬ 
tor  cortices  in  dogs  increases  with  conditioning  to 
a  visual  stimulus,  and  is  highest  at  the  time  of  the 
conditioned  stimulus  and  response.  John  et  al. 
(1973)  measured  the  relatedness  of  wave  forms 
recorded  from  multiple  brain  locations  of  cats  that 
were  learning  2  different  conditioned  responses. 
On  the  basis  of  wave  form  similarity,  he  demon¬ 
strated  differential  generalization  of  neural  activ¬ 
ity.  Bressler  (1987)  reported  that  correlations  be¬ 
tween  the  gamma  band  of  the  olfactory  bulb  and 
cortex  in  rabbits  are  specific  to  particular  regions 
of  each. 

In  studies  of  human  performance.  Livanov 
(1977)  found  that  correlation  levels  among  scalp 
electrodes  were  higher  during  motor  and  mental 
activity  than  during  rest.  Callaway  and  Harris 
(1974)  used  a  mutual  information  approach  to 
successfully  measure  the  degree  of  relatedness  be¬ 
tween  cortical  areas  during  mental  tasks.  Our  own 
work  with  human  subjects  to  date  has  been  aimed 
at  improving  the  precision  of  spatial,  temporaL 
and  behavioral  measurements  by  using  more  mod¬ 
ern  recording  and  signal  processing  technologies. 
In  one  study  (Gevins  et  al.  1981),  distinct,  com¬ 
plex,  and  rapidly  changing  patterns  of  evoked 
brain  potential  correlation  distinguished  sets  of 
single  trials  of  several  visuomotor  tasks  that  dif¬ 
fered  only  in  type  of  spatial  or  numerical  judg¬ 
ment.  In  a  later  study  of  a  brief  spaual  judgment 
task  (Gevins  et  al.  1983.  1985),  the  site  of  maxi¬ 
mum  differences  between  the  correlations  of  re¬ 
sponse  and  no-response  evoked  potentials  shifted 
during  sequential  stimulus-locked  intervals,  in  a 
manner  consistent  with  neuropsychological  mod¬ 
els  of  the  task. 

Major  methodological  improvements  over  our 
previous  studies  are  reported  here.  More  robust 
and  complete  methods  for  measuring  event-re¬ 
lated,  between-channel  covariation  were  devel¬ 
oped  and  applied  to  discern  differences  in  each 


stage  of  a  4  sec  long  task.  The  methods  included: 
(a)  the  use  of  laplacian  derivations  to  reduce  spa¬ 
tial  smeanng,  ( b )  rejection  of  oudying  trials  by  a 
pattern  classification  procedure,  (c)  interactive 
statistical  procedures  for  adjusting  the  distribu¬ 
tions  of  behavioral  variables  to  create  highly  con¬ 
trolled  data  sets,  and  (d)  analysis  of  inter-elec¬ 
trode  wave  form  covariance  and  time  delay  in 
numerous  brief  intervals. 

The  task  involved  cued  preparation,  stimulus 
evaluation,  response  execution  of  precise  right- 
and  left-hand  finger  pressures,  and  evaluation  of 
feedback  about  response  accuracy  (Fig.  1).  Spaual 
patterns  of  wave  form  covariance  and  timing  were 
derived  from  short  intervals  (Fig.  2)  of  averaged 
event-related  potentials  recorded  from  multiple 
scalp  electrodes.  Here  we  describe  our  methods 
and  their  application  to  stimulus-  and  response- 
locked  wave  forms,  averaged  across  subjects,  as  a 
test  of  whether  they  produce  results  consistent 
with  functional  neuroanatomical  models  found  in 
the  literature.  Part  II  (Gevins  et  al.  1989)  of  this 
paper  presents  the  prestimulus  and  feedback- 
evoked  results. 


General  methods  and  materials 

Seven  male  aduits  were  selected  for  participa¬ 
tion  in  this  study.  All  were  right-handed  according 
to  the  Edinburgh  Inventory  (Oldfield  1971).  None 
had  any  history  of  psychiatric  or  neurologic  dis¬ 
ease. 

Stimuli  for  this  experiment  were  presented  visu¬ 
ally  on  a  Videographics  II  amber  CRT  monitor, 
placed  70  cm  from  the  subject's  eyes,  and  sub¬ 
tended  a  visual  angle  of  <  1_5°.  Stimuli  had  a 
duration  of  315  msec  and  an  illumination  of  0.5 
log  fL  against  a  background  of  -  1J  log  fL.  One 
second  before  the  stimulus  appeared,  a  cue  (‘V’) 
was  presented  at  a  fixation  point  at  center  screen. 
The  cue  was  slanted  to  the  nght  or  left  to  indicate 
the  responding  hand  (Fig.  1).  In  ‘response-  trials 
(80%),  a  stimulus  number  from  1  to  9  was  pre¬ 
sented.  slanted  in  the  same  direction  as  the  cue; 
the  subject  was  to  respond  quickly  with  the  index 
finger  of  the  indicated  hand,  with  a  pressure  of 
0.1-0.9  kg,  according  to  the  stimulus  number.  A 
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RIGHT.  LEFT  INOEX  FINGER  -PRESSURE  TASK 
cue  stimulus  response  feedback 

ctr  *eau*"«o  3Su*c  **ow  *ccu«*rc» 

rO*  ca  «K>*r  «( SPOmSC  »*OOl>CE q 

a€5MN«  M(S9A( 

^  +  RIGHT  HAND  2.5 

2.5  UNITS 

^  -3  LEFT  HAND  3.3 

13  UNITS 

0  1  15  2  3 

Pig.  1.  Sequence  of  events  in  the  bimanual  visuomotor  task. 
The  nghi-cued.  right-hand  response  condition  is  shovm  above, 
while  the  left-cued,  left-hand  response  condition  is  shown 
below.  The  abscissa  shows  ume  in  seconds.  One  second  follow¬ 
ing  a  cue  i  a  slanted  letter  *V*),  a  slanted  1 -digit  stimulus  was 
presented.  If  the  direction  of  slant  (right  or  left)  of  the  stimulus 
number  was  the  same  as  the  cue.  the  subject  had  to  produce  a 
precise  pressure  with  his  right  or  left  index  finger  (depending 
on  the  direction  of  slant)  in  proportion  to  the  magnitude  of  the 
stimulus  number.  One  second  following  completion  of  the 
response,  a  2-digu  feedback  number  indicated  the  finger  pres¬ 
sure  that  the  subject  actually  produced.  On  a  random  20%  of 
the  trials,  the  stimulus  number  was  slanted  in  the  opposite 
direction  from  the  cue  (not  shown).  On  those  ‘catch’  trials,  the 
subject  had  to  inhibit  his  finger  response. 


random  20^  of  the  trials  were  miscued.  In  these 
‘no-response’  trials,  the  stimulus  was  slanted  in 
the  direction  opposite  to  the  cue,  and  the  subject 
was  to  withhold  his  response. 

As  feedback  for  response  trials,  a  2-digit  score 
that  indicated  the  response  pressure  to  a  tenth  of  a 
unit,  was  presented  1  sec  after  the  peak  of  the 
response  pressure.  Response  error  was  the  dif¬ 
ference  between  the  response  pressure  and  the 
stimulus  number.  For  each  hand,  an  adaptive  er¬ 
ror  tolerance  was  determined  as  the  moving  aver¬ 
age  of  response  error  for  the  preceding  5  response 
trials.  If  the  response  error  was  less  than  the  error 
tolerance  level  the  feedback  number  was  under¬ 
lined  to  indicate  a  ‘win’  and  the  subject  earned  a 
bonus  of  about  5  cents.  This  was  implemented  to 
equalize  task  difficulty  across  the  session  and  be¬ 
tween  hands,  and  to  provide  an  index  of  current 
performance  level  for  each  hand.  If  response  onset 
was  longer  than  1.5  sec  the  feedback  ‘00’  was 
presented.  The  feedback  ‘OK’  was  presented  1.5 
sec  after  the  stimulus  when  the  subject  correctly 
withheld  his  finger  response  on  no-response  trials. 


If  the  subject  incorrectly  responded  to  a  miscued 
(no-response)  stimulus,  he  received  the  feedback 
‘XX’  and  was  penalized  10  cents. 

Trials  were  presented  in  blocks  of  17.  with  the 
task  types  (right  and  left,  response  and  no-re¬ 
sponse)  randomly  ordered,  except  that  the  first  2 
trials  were  never  ‘no-response’  trials.  The  subject 
initiated  a  block  by  pressing  a  button,  and  the  17 
trials  followed  automatically  at  1.5  sec  intervals 
After  each  block,  a  summary  of  the  subject’s  aver¬ 
age  response  accuracy  for  each  hand,  the  number 
of  incorrect  responses,  and  the  monetary  bonus 
earned  appeared  on  the  monitor. 

Each  subject  practiced  the  task  in  a  pre-record¬ 
ing  session  until  his  performance  error  reached  an 
asymptote.  He  then  performed  between  300  and 
1000  trials  in  a  recording  session  that  lasted  ap¬ 
proximately  5  h.  including  short  rest  periods.  The 
adaptive  error  tolerance,  as  well  as  post-hoc  proce¬ 
dures  described  below,  were  used  to  control  for 
learning  and  fatigue. 

EEGs  from  26  scalp  electrodes  referenced  to 
the  midline  aniero-panetal  (aPz)  electrode  (placed 
halfway  between  Pz  and  Cz),  vertical  and  horizon¬ 
tal  eye  movements  (EOGs),  and  potentials  from 
responding  flexor  digitori  muscles  (EMGs)  re¬ 
corded  with  bipolar  electrodes,  were  digitized  at 
123  samples/sec  from  0.75  sec  before  the  cue  to 
1.0  sec  after  feedback.  Approximately  8500  trials 
of  data  were  collected  from  the  7  subjects  and 
were  edited  by  2  independent  raters  to  remove 
artifacts  evident  on  EEG.  EOG.  and  EMG  poly¬ 
graph  channels.  Additional  trials  were  eliminated 
in  which  the  response  was  slow,  biphasic  or  de¬ 
layed  beyond  1215  sec,  or  in  which  there  was  some 
EMG  activity  in  the  non-responding  hand  or  in 
the  cue-to-sumulus  epoch.  No-response  trials  with 
EMG  activity  were  also  eliminated. 


Analysis 

Prior  to  computation  of  event-related  covari¬ 
ances  (ERCs),  analytic  procedures  were  employed 
to  reduce  spatial  blurring,  to  select  trials  with 
consistent  event-related  signals,  and  to  control  for 
task-irrelevant  factors.  First,  the  laplacian  oper¬ 
ator  was  applied  to  the  potential  distribution  from 
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the  electrodes  that  were  not  located  at  the  periph¬ 
ery  of  the  scalp.  Second,  a  trial-selection  method 
increased  the  signal-to-noise  ratio  of  the  subse¬ 
quent  averaged  event-related  potential  (ERP),  and 
accentuated  differences  between  the  pair  of  condi¬ 
tions  to  be  compared.  Third,  trials  with  extreme 
behavioral  values  were  eliminated  to  obtain  behav- 
iorally  balanced  trial  sets. 

Averaged  ERPs  were  formed  from  these  lapia- 
cian.  enhanced,  balanced  trials,  then  digitally 
filtered  and  appropriately  down-sampled.  Analysis 
windows  were  selected  to  span  the  major  ERP 
peaks  (Fig.  2).  Multilag  cross-covariance  functions 
were  computed  between  all  pairwise  (by  channel) 
combinations  of  the  averaged  ERPs  in  each 
window.  The  individual  steps  will  now  be  pre¬ 
sented  in  detail. 

Reducing  volume  conduction  distortion  using  the 
laplacian  derivation  (LD) 

Potentials  which  arise  from  localized  sources  in 
the  brain  become  considerably  blurred  as  they  are 
volume-conducted  through  the  skull  and  scalp. 
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Fig.  2.  A  representative  averaged  wave  form,  from  the  midline 
parietal  channel  (Pz)  of  1  subject,  showing  the  major  ERP 
components  of  this  task  and  the  corresponding  event-related 
covariance  ( ERC)  analysis  intervals:  pre-stimulus  CNV,  post- 
stimulus  N1  and  F. 3.  response  N2R,  and  feedback  P3E  and 
P3L  The  response  onsets  of  the  individual  trials  forming  the 
average  varied,  and  the  average  was  time-locked  to  the  onset  of 
the  cue;  the  resulting  time  smear  attenuated  the  ERP  compo¬ 
nents  of  the  response  in  this  figure.  This  is  also  true  for 
feedback  components,  since  the  feedback  was  timed  to  the  end 
of  the  response.  This  was  not  a  problem  for  the  overall 
analysis,  since  separate  response  and  feedback  averages  were 
computed,  time-locked  to  their  respective  onsets. 


Consequently,  they  are  considerably  spread  by  the 
time  they  reach  the  scalp  (Rush  and  Driscoll 
1969).  In  order  to  reduce  this  smearing  and  to 
remove  the  effect  of  the  reference  electrode,  we 
used  the  LD  to  estimate  the  current  source  density 
from  the  scalp  potential  distribution.  This  tech¬ 
nique  compensates  for  the  spread  of  potential  and 
improves  the  spatial  resolution  of  scalp-recorded 
signals  by  approximating  the  local  spatial  second 
derivative  of  the  potential  field  at  a  point  on  the 
scalp.  This  derivative  is  proportional  to  the  den¬ 
sity  of  current  exiling  or  entenng  the  scalp  at  that 
point  (Nunez  1981).  (Determining  the  actual  cur¬ 
rent  would  require  knowledge  of  the  local  resistivi¬ 
ties  of  CSF,  skull,  and  scalp.) 

Hjorth  (1975.  1980)  originally  proposed  a  for¬ 
mulation  of  the  LD  at  a  particular  electrode  site 
using  surround  electrodes  at  hypothetically  uni¬ 
form  distances  and  at  right  angles.  Since  inter- 
electrode  distances  can  vary  considerably  between 
people,  we  improved  the  original  Hjorth  technique 
by  measuring  the  distances  between  electrodes,  as 
did  Thickbroom  et  al.  (1984),  who  proposed  the 
formulation 


where  fLD(i)  is  the  LD  of  electrode  i.  V,  is  the 
voltage  at  electrode  j,  d-  is  the  distance  between 
electrodes  i  and  j,  and  S;  is  the  set  of  electrodes 
surrounding  eiectrode  i.  Electrode  positions  were 
measured  in  3  dimensions  using  a  3-D  digitizer 
before  and  after  each  recording  session  (Gevms 
1988).  The  distances  d ^  were  the  ellipsoid  arc 
lengths  between  pairs  of  electrodes.  We  have  re¬ 
cently  improved  this  technique  further  by  using 
the  exact  angles  of  surround  electrodes  as  weil 
(Greer  and  Gevms  in  prep.). 

We  excluded  the  10  electrodes  at  the  periphery 
of  the  recording  montage  from  further  analysis 
because  there  was  inadequate  sampling  of  the 
surrounding  potential  at  these  sues.  (Representa¬ 
tion  of  the  spatial  second  derivative  of  potential  at 
a  point  depends  on  measurement  of  the  change  of 
potential  in  2  orthogonal  directions  at  the  surface.) 
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Examination  of  the  LD  wave  forms  confirmed 
that  the  signai-to-noise  ratios  were  low  (poor  sig¬ 
nal  representation)  at  these  peripheral  sites  com¬ 
pared  to  the  other  sites.  Sixteen  non-peripheral 
electrodes  remained:  left  and  right  frontal  (F3. 
F4),  midline  frontal  (Fz).  left  and  right  antero- 
central  (aC3,  aC4),  midline  antero-central  (aCz), 
left  and  right  central  (C3,  C4),  midline  central 
(Cz).  left  and  right  antero-parietai  (aPl,  aP2), 
midline  antero-panetal  (aPz).  left  and  right  parietal 
(P3.  P4).  midline  parietal  (Pz)  and  midline 
antero-occipital  (aOz).  The  antero-central  elec¬ 
trodes  were  halfway  between  the  corresponding 
central  and  frontal  sues,  the  antero-parietai  elec¬ 
trodes  were  halfway  between  the  parietal  and 
central  sites,  and  the  antero-occipital  electrodes 
were  halfway  between  the  occipital  and  parietal 
sites. 

The  peaks  in  LD  topographies  were  at  different 
locations  than  the  ERP  topographies  and  were 
more  spatially  localized  (cf..  Fig.  3A  and  B).  In 
most  cases,  this  localization  was  valuable  in  dis¬ 
tinguishing  overlapping  components.  The  polarity 
of  most  event-related  LD  peaks  changed  across 
the  scalp,  in  correspondence  to  sources  and  sinks 
(regions  of  emerging  and  entering  'equivalent  cur¬ 
rent’).  Thus,  they  were  labeled  by  the  standard 
ERP  terminology  (e.g.,  Nl)  according  to  their 
latency. 

Eliminating  trials  that  lacked  detectable  event-re¬ 
lated  signals 

We  developed  a  simple  method  for  choosing 
and  averaging  onlv  those  trials  that  had  detectable 
task-related  signals  (Gevins  et  al.  1986).  Rather 
than  selecting  trials  that  conformed  to  a  fixed 
model  of  the  shape  of  the  event-related  signal 
(e.g..  matched  filtering  or  'Weiner’  filtering  —  see 
reviews  in  Gevins  1984;  McGillem  and  Aunon 
1987),  we  used  a  pattern  classification  algorithm 
to  select  trials  that  were  statistically  different  from 
noise.  Trials  that  could  not  be  distinguished  from 
noise  were  not  included  in  the  averaged  wave 
forms. 

A  set  of  event-related  data  consisted  of  EEG 
segments  from  a  brief  time  interval  (125  or  250 
msec),  time-locked  to  the  event  of  interest.  This 
set  was  compared  to  a  set  of  noise  data,  composed 


of  EEG  segments  of  the  same  length,  but  ran¬ 
domly  timed  with  respect  to  stimulus  and  re¬ 
sponse  events.  The  distributions  of  the  event-re¬ 
lated  and  noise  sets  had,  on  the  average,  similar 
statistical  properties.  Therefore,  a  sophisticated 
mathemadcal  pattern  classification  algorithm  was 
needed  to  discriminate  the  2  sets  based  on  the  fact 
that  the  event-related  segments  were  non-sta- 
tionarv  and  the  noise  segments  were  stationary 
(Gevins  and  Morgan  1986). 

The  algorithm  constructed  equations  that  dis¬ 
criminated  the  2  sets  on  a  channel-by-channel 
basts.  Averaged  ERPs  for  each  channel  were 
formed  from  trials  containing  event-related  seg¬ 
ments  that  were  correctly  discriminated  from  noise. 
'Selected  trial  sets’  for  each  subject  were  formed 
from  trials  containing  segments  that  were  di¬ 
rectly  discriminated  from  ‘noise’  in  the  majority 
of  channels  with  significant  (PcO.Ol)  event-re¬ 
lated  signals. 

The  procedure  was  performed  for  individual 
subjects  and  channels,  using  low-pass  filtered  data 
(17-point  linear-phase  finite-impulse  response.  3 
dB  attenuation  at  7  Hz.  20  dB  attenuation  at  16 
Hz),  down-sampled  from  128  to  32  Hz.  Trial 
selection  of  stimulus-locked  data  used  a  250  msec 
wide  interval  centered  375  msec  after  the  stimulus. 
Trial  selection  of  response-locked  data  used  a  125 
msec  wide  interval,  centered  on  the  peak  of  the 
response-related  wave  form  for  each  subject.  This 
peak  latency  vaned  from  38  to  86  msec  after 
response  onset,  as  measured  from  the  finger-pres¬ 
sure  transducer  channel. 

Controlling  for  irrelevant  between-condinon  vari¬ 
ables 

It  was  important  to  balance  the  data  set  from  2 
conditions  for  stimulus-,  response-  or  perfor¬ 
mance-related  variables  that  were  different  from, 
but  unrelated  to.  the  intended  comparison.  For 
example,  it  was  important  that  the  pressure  of  left- 
and  right-handed  responses  did  not  differ.  For 
each  beiween-condition  comparison  for  each  sub¬ 
ject.  the  2  sets  of  event-related,  signal-beanng 
trials  were  statistically  balanced  by  eliminating 
trials  that  had  outlying  values  of  irrelevant  vari¬ 
ables  (Gevins  et  al.  1981,  1985).  This  procedure 
involved  an  interactive  program  that  displayed  the 
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table  t 

Numoers  of  original  and  final  trials  for  stimulus-locked  and  response-locked  data  (averaged  across  ’  subjects). 

Stimuius-locked  Response-locked 

Right-cued  Lett-cued  Right-cued  Lett-cued 

Response  No-response  Response  No-response 

Original  trials  1345  539  13S4  5u7  1395  1-19 

Final  trials  790  354  717  339  1079  906 


means.  Student’s  t  tests,  and  histograms  of  about 
50  behavioral  variables,  such  as  response  time, 
pressure,  velocity,  acceleration,  duration  and  er¬ 
ror.  as  well  as  stimulus  parameters,  indices  of 
muscle  activity  and  "arousal’  (integrated  energy 
measured  from  the  Pz  electrode,  and  computed  in 
500  msec  epochs  before  and  after  the  onset  of 
each  event).  Between-condiuon  balance  was 
achieved  when  the  Student’s  t  test  for  each  van- 
able  of  companson  had  a  significance  of  0.2  or 
greater.  "  Enhanced’  averages  of  these  event-related 
time  senes  were  then  computed  for  each  condition 
from  the  LD.  signal-bearing,  balanced  trial  sets  of 
each  subject. 

For  stimulus-locked  wave  forms,  data  sets  of 
right-  and  left-hand  responses  were  balanced  for 
pressure,  duration,  velocity,  acceleration  (when 
possible),  and  error  of  response.  Data  sets  of  each 
individual  subject  were  pruned  of  trials  that  had 
high  reaction  times  ( >  3  S.D.s),  but  were  not 
balanced  between  hands  because  of  inherent  hand 
differences.  No-response  (miscued)  sets  were  bal¬ 
anced  with  the  corresponding  response  sets 
(according  to  cued  hand)  by  pruning  no-response 
trials  that  contained  outlying  indices  of  prestimu¬ 
lus  "arousal.’  No-response  sets  were  then  compared 
between  hands  and  did  not  require  further  balanc¬ 
ing.  The  numbers  of  trials  in  the  original  and  final 
data  sets  are  given  in  Table  I. 

Response-locked  sets  for  each  subject  were 
pruned  of  trials  with  reaction  times  >  3  S.D.s  and 
balanced  between  hands  for  the  other  response 
variables  listed  above.  Reaction  time  and  EMG 
indices  were  not  balanced,  because  of  hand 
differences  within  subjects,  but  mean  reaction  time 
across  the  7  subjects  was  almost  the  same  between 
hands  (nght  hand  mean  —  611  ±136  msec,  left 
hand  mean  ■*  619  —  137  msec).  The  onset  of  the 


EMG  occurred,  on  the  average,  60  msec  before 
initiation  of  the  response.  The  numbers  of  original 
and  final  trials  are  given  in  Table  I. 

Beraeen-channel  covariance  measures 

The  presumulus  events  of  enhanced  averages 
were  bandpass  filtered  with  a  delta  band  filter  I  for 
the  low- frequency  contingent  negative  variation) 
and  the  other  events  were  filtered  with  a  theta 
band  filter  (Fig.  3C).  Each  filter  was  a  gaussian 
FIR  type,  with  no  side  lobes  in  the  time  response 
for  the  delta  case,  and  1  side  lobe  for  the  theta 
case.  This  lobe  limited  the  time  smear  to  the 
minimum  possible  smear  for  each  passband  and. 


A  0  C 


Fig.  3.  A:  stimulus-locked  averages  (of  all  7  subjects:  N  -  790) 
of  potentials  from  midline  electrode  sites  under  right-hand 
response  condiuons.  B:  unfiltered  laplaaan  denvauons  (LDs). 
C:  theta  band  filtered  LDs.  Positive  potential  is  up  for  A; 
emerging  equivalent  current’  is  up  for  B  and  C.  The  term  ij  in 
the  units  for  the  LD  indicates  that  for  each  electrode,  the 
voltages  at  the  neighboring  electrodes  were  weighted  by  the 
mean  of  the  to  those  neighbors.  The  posi-sumulus 

N1  is  shown  for  the  midline  antenxxsrpital  (aOz)  electrode. 
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in  particular,  ensured  that  analysis  windows  did 
not  include  major  contributions  from  other  peaks. 
Also,  since  the  filters  were  zero  phase,  they  did 
not  introduce  any  time  shifts.  Nonetheless,  we 
compared  filtered  and  unfiltered  averages  to  en¬ 
sure  that  the  filtering  did  not  differentially  distort 
the  relative  amplitudes  and  latencies  of  the 
analyzed  peaks. 

Multi-lag  covariance  functions  were  computed 
for  the  enhanced,  filtered  averages  (Fig.  4).  The 
covariance  analysis  interval  used  for  computing 
each  covanance  function  was  the  width  of  1  period 
of  the  band-center  frequency  of  each  filter. 
Down-sampling  factors  were  determined  by  the  20 
dB  rejection  point,  and  covariance  functions  were 
computed  up  to  a  lag  time  of  one-half  period  of 
the  high  frequency  for  each  band.  The  interval 
length  for  the  theta  (4—7  Hz)  band  was  187  msec, 
corresponding  to  one  period  at  5.5  Hz.  The  theta- 
filtered  data,  originally  sampled  at  128  Hz.  were 
down-sampled  by  6,  and  the  covariance  function 
computed  to  8  lags  ( ±  62  msec).  The  interval 
length  for  the  delta  (0.1-3  Hz)  band  was  375 
msec,  corresponding  to  one  period  at  2J  Hz. 
Delta- filtered  data  were  down-sampled  by  8,  and 
the  covariance  function  computed  to  16  lags  ( ±  125 
msec).  The  covariance  measure  was  defined  as  the 
maximum  absolute  value  of  the  covariance  func¬ 
tion.  and  the  time  delay  as  the  lag  time  of  that 
maximum.  (Note  that  the  covariance  is  not  nor¬ 
malized  by  signal  strength  and  is  therefore  sensi¬ 
tive  to  the  amplitude,  as  well  as  to  the  shape,  of 
time  series  from  the  2  channels.) 

Covanance  measures  were  converted  to  signifi¬ 
cance  scores  (after  square  root  transformation  was 


used  to  approximately  normalize  the  signal- 
squared  covariance  distribution)  by  comparing 
them  with  the  noise  median  and  with  Tukey’s 
biweight  estimate  of  noise  variance  (Igiewicz  1983). 
Tukey’s  biweight  estimate  was  computed  from 
sample  distributions  of  zero-lag  covariances  be¬ 
tween  intervals  centered  around  data  points  with  a 
minimum  energy  envelope.  For  N  data  points,  the 
covariances  have  N  —  1  degrees  of  freedom.  Thus 
the  theta  band  covariances  were  computed  from  5 
down-sampled  time  points,  and  had  4  degrees  of 
freedom.  Despite  this  low  value,  significances  fre¬ 
quently  reached  P  <  0.00001.  Adjustment  for  mul¬ 
tiple  comparisons  within  an  interval  (120  pairings 
of  16  non-peripheral  channels)  was  done  by  a 
Duncan  procedure,  in  which  the  number  of  chan¬ 
nels  was  taken  as  a  conservative  approximation  of 
the  number  of  independent  covariance  pairs. 

To  test  the  mean  difference  between  the  ERC 
patterns  under  2  conditions,  the  Student’s  t  test 
was  applied  to  the  distributions  of  all  significant 
ERCs  for  each  condition.  Similarity  between  2 
ERC  patterns,  irrespective  of  the  magnitudes  of 
the  ERCs.  was  measured  with  an  estimate  of  the 
correlation,  and  its  confidence  interval  was  pro¬ 
duced  by  the  distribution-independent  ‘  bootstrap’ 
Monte  Carlo  procedure  (Efron  1982).  This  gener¬ 
ated  an  ensemble  of  correlation  values,  each  com¬ 
puted  for  a  sample  of  the  ERC  distribution  formed 
by  random  selection  with  replacement. 

To  show  the  most  prominent  results,  all  signifi¬ 
cant  ERCs  within  1  S.D.  from  the  maximum  value 
in  that  interval  were  displayed  (Fig.  4).  Diagrams 
of  ERCs  within  2  S.D.s  were  also  examined  for 
additional  information.  Both  time  senes  contrib- 


Fig.  4.  A:  schematic  diagram  on  a  top  view  of  a  model  head,  showing  the  relationship  of  an  ERC  line  (left)  to  the  ERP  wave  forms 
from  the  2  corresponding  electrode  sites  (right).  The  ERC  was  computed  over  the  indicated  187  msec  analysis  interval  from  the  theta 
band-filtered.  averaged,  response-locked  LD  ERP  segments.  In  the  ERC  diagram,  the  width  of  a  line  connecting  2  sites  indicates  the 
significance  of  the  covanance  between  the  ERP  wave  forms  from  those  sites,  with  the  scale  appeanng  above  the  word  'significance.' 
The  color  of  the  line  indicates  the  ume  delay  in  msec  (lag  ume  of  maximum  ERC)  as  shown  in  the  scale  above  'msec  delay.'  The 
color  of  the  arrow  indicates  the  sign  of  the  ERC  (same  color  as  line  -  positive:  tan  -  negative).  The  arrow  points  from  the  leading  to 
the  lagging  channel,  unless  there  is  no  delay,  in  which  case  a  bar  is  shown.  The  ERC  between  aPz  and  aCz  is  significant  at  P  <  10" s. 
The  aPz  wave  form  leads  the  aCz  wave  form  by  about  16-31  msec  (green  line),  and  the  ERC  is  posiuve  (arrow  also  green).  B: 
schematic  diagram  showing  ail  possible  ERC  lines  between  pairs  of  the  16  non-penpheraJ  electrode  sues.  All  26  electrode  sues  are 
indicated  as  red  disks,  but  computauon  of  an  adequate  estimate  of  the  LD  was  not  actually  possible  at  the  10  peripheral  sites.  Since  a 
ring  of  peripheral  electrodes  must  be  expended  in  order  to  estimate  the  LD  at  the  interior  sues,  there  are  soil  large  unrecorded  scalp 

areas  even  when  recordings  are  made  with  26  electrodes. 
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uting  to  each  significant  ERC  were  examined  to 
ensure  that  the  ERC  was  due  to  the  primary  ERP 
component,  and  not  to  any  interactions  resulting 
from  the  unavoidable  time  smear  caused  by  nar¬ 
row-band  filtering  (Fig.  3Q.  The  most  prominent 
electrode  site  was  the  one  that  covaried  signifi¬ 
cantly  with  the  most  other  sites.  Other  prominent 
sites  covaried  with  more  than  one-half  the  number 
of  sites  of  the  most  prominent  site. 


Results 


Stimulus-locked  wave  forms 

The  unfiltered  LD  response  and  no-response 
wave  forms,  averaged  over  all  7  subjects,  con¬ 
tained  peaks  at  different  sites;  these  peaks  corre¬ 
sponded  to  Nl  at  130-140  msec  and  P2  at  230-240 
msec  (Fig.  5).  The  entering  current,  corresponding 
to  Nl.  was  greatest  at  the  lateral  parietal  sites  (not 
shown).  There  were  2  sites  of  maximal  exiung 
current:  the  midline  antero-central  and  midline 
antero-parietal  sites,  corresponding  to  the  anterior 
and  posterior  fail-off  of  the  potential  field.  It  is 
likely  that  Nl  at  the  anterior  sites  involved  resolu¬ 
tion  of  the  contingent  negative  variation  (CNV; 
Tecce  1972). 

Right-  and  left-hand  response  wave  forms  con¬ 
tained  a  P3  wave,  approximately  250  msec  wide, 
centered  near  350  msec,  with  maximal  emerging 
current  at  Pz  (Fig.  5,  left).  The  infrequent  no-re¬ 
sponse  wave  forms  had  a  much  larger  P3  wave, 
with  maximal  emerging  current  at  aCz  (Fig.  5, 
right).  The  response  P3  wave  had  the  posterior 
maximum  characteristic  of  the  task-relevant  ‘P300’ 
peak  reported  in  the  literature  (e.g.,  Squires  et  al. 
1977).  The  no-response  P3  had  the  larger  ampli¬ 
tude  characteristic  of  the  probability-sensitive 
‘P300,’  and  the  anterior  maximum  characteristic 
of  the  ‘no-go’  P3a  (Squires  et  al.  1975). 

In  response  wave  forms,  a  slow  shift  corre¬ 
sponding  to  response  preparation  began  after  P2, 
and  peaked  at  about  600  msec  (Fig.  5,  left).  Enter¬ 
ing  current  was  largest  at  aCz,  with  larger  ampli¬ 
tude  at  central  sites  contralateral  to  the  respond¬ 
ing  hand  (not  shown).  (For  the  right  response,  the 
amplitude  of  C3  was  approximately  9  times  greater 
than  the  amplitude  of  C4;  for  the  left  response. 
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Fig.  5.  Right-hand  response  (A.  N  -  790)  and  nght-hand  no-re¬ 
sponse  (B.  N  -  354)  averaged,  unfiliered.  7-subject.  stimulus- 
locked.  LD  wave  forms  from  midline  electrode  sites.  The  Nl 
peak  and  corresponding  ERC  analysis  interval  are  shown  for 
the  aOz  electrode,  the  P2  peak  is  shown  for  the  Pz  electrode, 
and  the  P3  peak  and  interval  are  shown  for  the  aCz  (midline 
antero-central)  electrode.  The  difference  between  conditions  is 
highlighted  by  the  large  P3  peak  (maximal  at  aCz)  in  the 
infrequent  no-response  condition.  Emerging  "equivalent  cur¬ 
rent’  up. 


the  amplitude  of  C4  was  about  2.5  times  greater 
than  the  amplitude  of  C3.)  In  no-response  trials,  a 
broad  ‘slow  wave'  peaked  at  550  msec  at  Fz  and 
700  msec  at  aCz.  then  extended  past  1  sec  (Fig.  5. 
right).  The  slow  wave  had  large  entering  current  at 
midline  central  and  frontal  sites  (maximal  at  aCz), 
but  was  small  elsewhere. 

Theta  filtering  emphasized  the  early  peaks,  in¬ 
cluding  the  no- response  anterior  P3  wave,  and 
eliminated  the  slower  response  preparation  and 
slow  wave  components.  It  also  resolved  the  wide 
posterior  P3  wave  into  2  theta  band  peaks. 


EVENT-RELATED  COVARIANCES  STIMULUS  AND  RESPONSE 


Sumulus-iocked  ERCs 

ERCs  were  computed  for  the  theta  band-filtered 
data  in  a  senes  of  1ST  msec  wide  intervals  centered 
before  and  after  the  stimulus.  ERC  patterns  were 
highly  consistent  over  the  early  intervals,  reflect¬ 
ing  the  high  degree  of  synchrony  of  the  filtered  N1 
and  P2  peaks.  In  a  representative  interval  centered 
62  msec  after  the  stimulus  (Fig.  6).  there  was  a 
common  pattern  for  both  the  right  and  left  re¬ 
sponse  stimulus  conditions  at  the  threshold  of  1 
S.D.  It  consisted  of  ERCs  of  the  midline  antero- 
parietal  site  with  midline  panetal  and  antero- 
central  sites.  In  these  early  intervals,  the  right- 
slanted  stimulus  patterns,  but  not  the  left,  also 
involved  the  nght  parietal  site  (P4).  At  2  S.D.s 
from  the  maximum  ERC.  however,  the  left  stimu¬ 
lus  patterns  included  both  P3  and  P4.  whereas  the 
right  stimulus  patterns  included  P4  and  aC3.  but 
not  P3.  In  these  early  intervals,  the  patterns  for 
response  and  no-response  conditions  were  also  the 
same. 

In  order  to  test  the  significance  of  the 
between-condition  difference  observed  in  the  pat¬ 
terns  of  Fig.  6.  we  used  the  Student’s  t  test  to 
compare  the  distributions  of  all  the  significant 
ERCs  ( P  <  0.05)  of  the  right  and  left  stimulus 
patterns.  The  distributions  of  right  and  left  stimu¬ 
lus  ERC  patterns  were  significantly  different  at 
P  <  0.05  ( t  =  2.1.  df  —  238).  The  bootstrap  corre¬ 
lation  was  0.86  ±  0.02.  Thus,  although  the  distri¬ 
butions  were  significantly  different  in  overall  scale, 
they  had  a  similar  underlying  pattern. 

The  ERC  patterns  of  response  and  no-response 
trials  diverged  by  the  interval  centered  at  281 
msec  (not  shown).  As  characterized  by  the  312 
msec  centered  pattern  (Fig.  7,  right),  the  Pz  site 
was  prominent  in  the  response  patterns  and  had 
significant  ERCs  with  right  parietal  and  antero- 
parietal.  midline  and  left  antero-cemral  and  right 
frontal  sites.  The  midline  antero-central  site  was 
prominent  in  the  no-response  patterns  and  co¬ 
varied  with  Pz,  F3,  Fz,  and  aC4  (Fig.  7.  left).  The 
aCz-Fz  EFC,  with  aCz  leading  by  48-79  msec, 
was  particularly  characteristic  of  the  no-response 
patterns  during  the  intervals  from  281  to  375 
msec.  The  difference  between  the  distributions  of 
all  significant  ERCs  ( P  <  0.05)  that  corresponded 
to  the  response  and  no-response  patterns  observed 


in  Fig.  7  was  highly  significant  at  P  <  0.001  (/  = 
11.3.  df  =  238).  The  bootstrap  correlation  was  0.5') 
±  0.02.  Therefore,  the  scale  difference  between  the 
response  and  no-response  distributions  was  highly 
significant,  but  they  had  some  common  ERCs. 

The  response  patterns  underwent  another 
change  at  the  375  msec  centered  interval,  as  the 
intervals  approached  the  onset  of  the  response. 
The  patterns  became  more  complex,  with  more 
ERCs  involving  more  electrode  sites.  By  the  500 
msec  interval,  the  right-  and  left-hand  response 
patterns  involved  the  same  sites,  with  prominent 
involvement  of  aCz.  The  no-response  patterns  did 
not  undergo  any  major  changes  from  the  281  to 
500  msec  centered  intervals  except  that  they  di¬ 
minished  in  complexitv.  By  500  msec,  the  only 
significant  no-response  ERC  (at  either  the  1  or  2 
S.D.  thresholds)  was  aCz-Fz.  with  aCz  leading  bv 
48-79  msec  for  both  hands. 

Response-locked  wave  forms 

The  unfiltered  LD  wave  forms  (Fig.  8)  con¬ 
tained  a  slowly  increasing  shift  that  corresponded 
to  the  readiness  potential  (RP).  It  was  maximal 
before  the  onset  of  the  response  and  was  char¬ 
acterized  by  an  increase  in  slope  that  peaked  62 
msec  after  the  response  began  (‘N2R’).  and  by  a 
component  that  peaked  187  msec  after  the  re¬ 
sponse  began  (‘P2R’).  The  N2R  peak  was  local¬ 
ized  to  only  a  few  midline  sites,  and  to  sites 
contralateral  to  the  responding  hand.  Current  sinks 
were  anterior  (maximum  at  aCz).  Current  sources 
were  posterior:  the  only  sources  were  C3.  aPl  and 
P3  for  the  right  hand,  and  the  homologous  right- 
hemisphere  sites  for  the  left  hand. 

Left-  and  right-hemisphere  maxima  were  at  aPl 
and  aP2.  respectively;  therefore,  there  was  an  an¬ 
terior-posterior  inversion  of  polarity,  but  no  inver¬ 
sion  along  the  midline. 

The  P2R  component  was  present  at  only  a  few 
electrode  sites;  it  was  absent  at  aCz.  where  the  RP 
and  N2R  were  strongest.  For  the  right-hand  re¬ 
sponse.  the  maximum  current  sink  was  at  C3.  and 
principal  sources  were  at  Cz  and  F3.  For  the 
lei.  hand  response,  the  maximum  sink  was  at  C4; 
the  sources  were  at  Cz.  F4.  and  aC4.  Theta  band 
filtering  removed  the  RP  and  enhanced  N2R  and 
P2R. 
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Response-locked  ERCs  response-locked  averaged  LD  wave  form  before 

ERCs  were  computed  for  a  series  of  187  msec  and  after  onset  of  the  response.  In  the  interval 

wide  intervals  centered  on  the  theta  band-filtered.  centered  187  msec  before  the  response,  the  ERC 
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Fig.  o.  View  of  tne  most  significant  (wuhin  1  S.D.  of  largest 
value)  ERCs  from  ihe  siimuius-locked.  LD  N1  wave  of  the 
response  condition,  superimposed  on  colored  maps  of  N1  wave 
amplitude.  ERCs  were  measured  during  a  187  msec  wide 
interval.  centered  62  msec  after  left  (left)  and  right  (right) 
stimulus  onset,  from  the  theta  band-filtered  averages  over  ail  7 
subjects.  The  ERC  diagram  conforms  to  the  significance,  sign, 
and  deiay  scales  of  Fig.  FA.  The  color  scale  at  the  left, 
representing  wave  amplitude,  covers  the  range  from  the  minimal 
to  maximal  values  of  the  2  maps.  For  both  left-  and  right- 
handed  conditions,  there  is  a  maximum  of  wave  amplitude 
occipitally  and  a  minimum  at  the  midline  antero-centrai  (aCz) 
site.  Common  ERCs  for  both  left-  and  nght-hand  conditions 
exist  between  midline  parietal  (Pz)  and  antero-panetal  (aPz). 
and  between  midline  aniero-oanetal  and  antero-centrai  sites. 
Additional  ERCs  for  the  ngni-hand  condition,  involving  the 
nght  panetai  iP4)  site,  reflect  the  lateralization  of  the  N1  peak 
at  panetai  sites. 

Fig.  '.  View  of  the  most  significant  itop  standard  deviation)  Pc 
ERC  patterns  supenmposed  on  colored  maps  of  integrated  P3 
amplitude.  Measurements  were  made  from  theta  band-filtered 
averages  over  all  7  subjects,  dunng  an  interval  218—105  msec 
after  onset  of  the  nght-cue  stimulus,  for  pressure  response 
(nght)  and  no-response  deft)  conditions.  The  midline  panetai 
sue  is  prominent  in  the  pattern  for  the  response  condition,  and 
the  midline  antero-centrai  site  is  prominent  in  the  infrequent 
no-response  condition.  Pz  iags  most  other  sites  in  the  response 
condition,  while  aCz  lags  in  the  no-response  condition. 
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Fig.  8.  Response-locked,  unfiltered.  LD  wave  forms  of  lateral 
electrode  sites  for  left-  and  ngnt-nand  index  finger  responses. 
The  RP  (readiness  potential).  N2R'  and  P2R'  components 
are  indicated,  as  .s  the  N2R  ERC  interval.  Note  the  reversals 
of  poianty  in  N2R  and  PTR  between  C3  and  aC3  (ngnt  hand), 
and  between  C4  and  aC-i  left  hand).  Emerging  equivalent 
current'  up. 


Fig.  9.  View  of  the  most  significant  (top  standard  deviation)  ERCs  from  the  N2R  wave  supenmposed  on  colored  maps  of  the 
amplitude  of  that  wave  for  left-  (left.  N  -  906)  and  right-hand  (nght.  N  - 1079)  index  finger  responses.  N2R  ERCs  were  measured 
dunng  a  187  msec  interval  centered  on  the  peak  of  left-hand  and  nght-hand  index  finger  pressures  from  theta  band-filtered  averages 
over  all  7  subjects.  The  ERC  diagram  conforms  to  the  significance,  sign,  and  delay  scales  of  Fig.  4A.  The  maximal  wave  amplitude  is 
at  lateraiized  central  (C4  and  C3)  and  antero-panetal  (aP4  and  aP3)  sites,  while  the  minimal  amplitude  is  at  the  midline 
antero-centrai  (aCz)  site.  Note  that  the  aCz  site  is  involved  in  all  ERCs  in  this  interval.  The  patterns  are  distinctly  lateraiized 
according  to  responding  hand.  The  sign  of  ERCs  involving  aCz  is  positive  for  lateral  frontal  sites,  and  negative  for  lateral  central  and 

antero-panetal  sues.  Note  the  16-31  msec  lag  between  aCz  and  Fz. 
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patterns  were  weak  and  did  not  differ  appreciably 
between  hands.  The  ERC  patterns  became  strong 
and  contralateral  to  the  responding  hand  begin¬ 
ning  with  the  interval  centered  at  the  response 
onset. 

Right-  and  left-hand  patterns  within  the  thresh¬ 
old  of  1  S.D.  were  nearly  mirror  images  in  the 
N2R  interval  (Fig.  9).  centered  62  msec  after 
response  onset.  The  aCz  site  was  involved  in  ail 
ERCs  in  both  patterns.  This  site  covaried  with 
short  delay  (0-15  msec)  with  the  antero-parietal. 
central  and  frontal  sites  contralateral  to  the  re¬ 
sponding  hand  (and  with  the  left  antero-central 
site  for  the  right-hand  pattern),  and  with  the  mid¬ 
line  central  site.  Additionally,  aCz  led  Fz  by  16-31 
msec.  The  ERCs  of  aCz  with  aPl  and  C3  for  the 
right  hand,  and  with  aP2  and  C4  for  the  left,  were 
negative.  ERCs  were  positive  for  aCz  with  aC3 
and  F3  for  the  right  hand,  and  with  F3  for  the 
left.  Additional  ERCs.  which  were  included  at  the 
2  S.D.  threshold,  were  all  contralateral  to  the 
responding  hand.  The  only  other  sites  involved 
were  P3  for  the  nght-hand  pattern,  and  P4  and 
aC4  for  the  left.  Right-  a.  d  left-hand  patterns  in 
the  interval  centered  at  187  msec,  at  both  1  and  2 
S.D.s,  involved  the  same  sites  as  in  the  62  msec 
centered  interval.  The  lags  increased  from  the 
0-31  msec  to  the  16-47  msec  range. 

The  distributions  of  all  significant  response-re¬ 
lated  ERCs  during  the  62  msec  centered  interval 
differed  significantly  between  right  and  left  hands 
at  P  <  0.02  (/  =  2.6,  df  =  192).  The  bootstrap  cor¬ 
relation  wes  0.06  ±  0.12.  Thus,  not  only  were  the 
distributions  significantly  different  from  each 
other,  they  were  also  uncorrelated. 

Discussion 

This  study  suggests  that  the  method  of  event-re¬ 
lated  covariance  (ERC)  can  measure  signs  of  the 
dynamic  functional  conical  networks  underlying 
goal-directed  behaviors  at  the  scalp.  Indeed,  the 
distinct,  rapidly  changing  spatial  patterns  of  wave 
form  synchronization  seen  in  event-related  covari¬ 
ance  maps  correspond  to  clinical  neuropsychologi¬ 
cal  models  of  a  simple  numerical  judgment  task 
that  requires  visual  stimulus  processing  followed 
by  response  execution.  Furthermore,  having  found 


specific  task-related  time  differences  between  wave 
forms  within  a  given  analysis  interval  we  suggest 
that  functional  conical  networks  may  be  char¬ 
acterized  by  the  time  relations  of  acuvity  at  their 
nodes.  While  the  ERC  patterns  suggest  functional 
co-involvement  of  underlying  conical  regions,  we 
must  reiterate  that  the  actual  generators  are  in  fact 
unknown,  and  further  studies  will  be  needed  to 
determine  what  they  are.  With  this  caveat  in  mind, 
we  present  a  discussion  of  the  functional  neuro- 
anatomical  implications  of  the  results. 

Response-related  ERPs  and  ER  Cs 

Since  ERC  patterns  for  the  robust  N2R  and 
P2R  components  ot  response-related  wave  forms 
were  simple  and  clear  cut.  they  will  be  discussed 
first.  There  were  several  main  findings.  First,  the 
midline  antero-central  electrode  site  (aCz).  overly¬ 
ing  premotor  and  supplementary  motor  cortices 
thought  to  be  involved  in  planning  and  execution 
of  precise  finger  control  (Ingvar  and  Philipson 
1977;  Foit  et  al.  1980;  Roland  et  al.  1982),  was 
central  to  all  response-related  patterns.  Second, 
response-related  ERC  patterns  reflected  an  inver¬ 
sion  of  polarity  in  the  filtered  LD  wave  forms 
between  the  lateralized  central  and  the  Iateraiized 
antero-central  and/or  frontal  sites.  The  polarity 
reversal,  possibly  extending  across  the  central 
sulcus,  may  reflect  a  single  dipole  source  generator 
in  the  finger  region  of  the  motor  cortex.  However, 
if  this  were  the  case,  the  2  poles  of  the  generator 
would  be  highly  covariant,  that  is.  there  would  be 
high  covariance  between  lateral  antero-central. 
and  central  sites.  Since  this  was  not  so,  we  favor 
the  interpretation  that  there  are  2  closely  spaced 
generators,  1  motor  (antero-central)  and  1  somes- 
theiic  (post-central).  Third,  midline  ERCs  of  the 
antero-central  site  with  the  frontal  and  central 
sites  were  response  features  common  to  both 
hands.  The  ERC  between  the  antero-central  and 
central  sites  had  a  short-lag  delay  and  may  reflect 
volume-conducted  activity  from  a  single  genera¬ 
tor.  The  longer-lay  delay  (16-31  msec)  between 
the  antero-central  and  frontal  sites  suggests  the 
presence  of  2  distinct  generators,  although  ad¬ 
ditional  studies  are  required  to  rule  out  a  'rotat¬ 
ing'  single  generator  as  the  source  of  the  covari¬ 
ance  (Fender  198T). 
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The  findings  reported  here  thus  support  the 
conclusion  that  during  the  N2R  and  P2R  compo¬ 
nents  of  a  voluntary,  skilled,  ballistic  finger  re¬ 
sponse.  there  are  at  least  4  primary  cortical  gener¬ 
ators:  one  midline  in  premotor  and/or  supple¬ 
mentary  motor  cortices;  another  midline  in  pre¬ 
frontal  cortex:  a  third  in  the  hand  region  of  motor 
cortex  contralateral  to  the  responding  hand;  and  a 
fourth  in  the  contralateral  hand  region  of  the 
somesthetic  cortex.  These  areas  are  the  same  as 
those  determined  by  studies  of  direct  cortical  re¬ 
cordings  of  motor  potentials  in  humans  and 
monkeys  (Arezzo  and  Vaughan  1975:  Pieper  et  ai. 
1980).  although  prefrontal  involvement  was  absent 
in  monkeys. 

Stimulus-related  ERPs  and  ERCs 

The  stimulus-related  patterns  displayed  several 
characteristic  features.  For  example,  the  patterns 
of  intervals  within  the  first  250  msec  differed 
according  to  the  slant  of  the  stimulus.  There  were 
short-delav  ERCs  involving  the  midline  antero- 
panetal  site,  midline  antero-central  site,  and  mid- 
line  parietal  site  in  the  patterns  of  all  the  condi¬ 
tions.  This  suggests  that  perceptual  processing  in 
parietal  areas  was  coordinated  with  motor  ini¬ 
tiation  control  in  supplementary  motor  and/or 
premotor  areas  that  had  been  ‘primed’  for  re¬ 
sponse  by  the  cue. 

The  right  parietal  site  was  a  salient  feature  of 
the  ERC  patterns  for  the  nght-slanted  stimulus, 
regardless  of  cue.  For  the  left-slanted  stimulus, 
there  was  weak  involvement  of  both  the  right  and 
left  parietal  sites  (both  of  which  were  only  evident 
at  the  2  S.D.  threshold).  Therefore,  the  right 
parietal  sue  was  involved  regardless  of  the  stimu¬ 
lus  and  was  stronger  when  the  stimulus  was  slanted 
to  the  right.  The  left  parietal  site  was  involved 
when  the  stimulus  was  slanted  to  the  left.  The 
appearance  of  the  right  parietal  site  in  patterns  for 
both  right  and  left  hands  may  reflect  the  impor¬ 
tance  of  that  region  for  humans  performing  atten- 
tional  tasks  (Mesulam  1981).  Our  result  may  be 
compared  with  the  finding  of  Heilman  and  Van 
den  Abell  (1980)  that  the  right  parietal  lobe  is 
focally  activated  by  stimuli  projected  into  either 
visual  Field,  whereas  the  left  is  only  activated  by 
stimuli  projected  into  the  right  field.  The  fact  that 


the  component  that  changed  hemisphere  in  our 
ERC  patterns  was  ipsilaterai.  rather  than  con¬ 
tralateral  to  the  stimulus  direction,  may  be  ex¬ 
plained  by  a  contralateraily  activated,  but 
obliquely  oriented  generator  that  projects  its  field 
ipsilaterai  to  the  slant  of  the  stimulus.  Even  though 
the  stimulus  subtended  only  a  small  visual  angle, 
its  tilt  into  one-half  of  the  visual  field  may  have 
evoked  an  obliquely  oriented  generator  popula¬ 
tion.  possibly  buried  in  the  parieto-occipital  sulcus. 
Such  a  ‘paradoxical’  ipsilaterai  postenor  peak  of 
potential  in  response  to  foveal  visual  stimulation 
has  been  reported  by  Barrett  et  al.  (1976). 

The  ERC  patterns  began  to  differ  according  to 
the  type  of  response  after  the  early  intervals,  while 
differences  related  to  the  physical  charactenstics 
of  the  stimulus  disappeared.  By  the  interval 
centered  at  312  msec  after  the  stimulus,  which 
spanned  the  P3  wave,  the  ERC  patterns  clearly 
differed  according  to  the  subjective  interpretation 
of  the  stimulus,  rather  than  to  the  stimulus  per  se. 
The  difference  between  the  anterior  (aCz)  con¬ 
centration  of  ERCs  in  the  no-response  condition, 
and  the  posterior  (Pz)  concentration  in  the  re¬ 
sponse  condition,  is  consistent  with  other  studies 
of  go/no-go  tasks  (see  summary  in  Tueting  1978). 

There  has  been  considerable  interest  in  the 
neural  substrates  of  the  P3  component,  with  evi¬ 
dence  of  widespread  cortical  and  subcortical 
sources  (Halgren  et  al.  1980:  Wood  et  al.  1980: 
Okada  1983;  Glover  et  al.  1986;  Velasco  et  al. 
1986).  The  complex  pattern  of  covariance  and 
delay  for  the  P3  wave  in  the  infrequent  no-re- 
sponse  condition  suggests  that  more  than  one 
source  is  responsible  for  its  generation.  In  particu¬ 
lar,  the  long-lag  delays  (48-79  msec)  of  the  mid¬ 
line  antero-central  with  the  midline  and  left  fron¬ 
tal  sites,  as  well  as  the  concurrent  short  delays 
( <  31  msec)  of  the  midline  antero-central  with 
right  antero-central  and  midline  parietal  sites,  are 
difficult  to  reconcile  with  the  notion  of  a  single 
generator. 

The  ERC  between  Fz  and  aCz  was  prominent 
in  later  intervals  of  both  response  and  no-response 
stimulus-locked  patterns.  Examination  of  the 
filtered  wave  forms  showed  that  aCz  and  Fz 
changed  shape  in  the  period  from  375  to  500  msec 
post  stimulus,  so  that  the  aCz  site  led  Fz  with 
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short  delay  in  the  response  condition  and  with 
long  delay  in  the  no-response  condition.  This  ap¬ 
parent  condition-dependent  adjustment  in  the  tim¬ 
ing  of  the  wave  forms,  along  with  the  significance 
of  their  ERC,  suggests  that  the  relation  of  these  2 
sites  is  an  important  sign  of  the  subjective  deci¬ 
sion  to  execute  or  withhold  the  finger  press.  Since 
aCz  was  a  prominent  site  in  all  response  patterns, 
its  short-delav  timing  relation  to  Fz  may  reflect  a 
higher  order  cognitive  control  process  for  the  ini¬ 
tiation  of  response.  The  midline  frontal  electrode 
site  overlies  the  prefrontal  cortex,  which  is  known 
to  be  involved  in  the  temporal  organization  of 
behavior  (Ingvar  1985;  Stuss  and  Benson  1986). 

ERC  method:  advances,  remaining  issues 

The  methods  presented  here  represent  a  signifi¬ 
cant  advance  over  those  of  our  previous  studies 
(Gevtns  et  al.  1981.  1983,  1985),  in  that  we  have 
now  found  task-specific  scalp  patterns  of  mass 
neural  integration  that  conform  to  a  priori  predic¬ 
tions  based  on  the  presumed  function  of  underly¬ 
ing  cortical  areas.  It  is  unlikely  that  volume  con¬ 
duction  blurring  played  a  significant  role  in  the 
resulting  scalp  ERC  patterns.  First,  the  LD  greatly 
reduced  the  effect  of  blurring  from  the  spatial 
spread  of  potential.  Second,  because  we  consid¬ 
ered  only  the  most  significant  ERCs,  we  reduced 
any  global  effects  due  to  potential  spread.  Third, 
although  volume  conduction  cannot  be  ruled  out 
as  the  basis  for  ERCs  between  neighboring  sites. 
ERCs  due  to  volume  conduction  would  tend  to  be 
greatest  for  nearby  sites  and  would  decrease  uni¬ 
formly  with  distance  in  all  directions.  Volume- 
conduction  blurring  is  not  considered  to  be  a 
major  factor  in  determining  the  ERC  patterns 
since  the  patterns  presented  here  were  highly 
localized  and  involved  ERCs  between  sites  with 
non-covarying  intervening  sites. 

Although  we  cannot  rule  out  subcortical 
sources,  the  same  reasoning  makes  it  seem  un¬ 
likely  that  they  are  directly  responsible  for  the 
ERC  patterns.  Subcortical  sources  would  tend  to 
produce  diffuse  scalp  patterns  in  which  ail  inter¬ 
vening  sites  covaried.  We  should  emphasize,  how¬ 
ever,  that  even  if  all  the  sources  of  the  scalp  ERC 
patterns  are  cortical,  the  ERCs  do  not  necessarily 
result  from  direct  communication  between  conical 


regions.  There  are  numerous  pathways,  involving 
subcortical  and  other  cortical  structures,  that  could 
account  for  the  measurement  of  similar  wave  forms 
from  any  2  cortical  regions. 

Determination  of  the  actual  sources  of  these 
ERC  patterns  awaits  further  studies  using  com¬ 
bined  electrical,  magnetic,  and  radiological  data, 
as  well  as  intracerebral  data  from  human  patients 
and  non-human  primates.  Since  these  studies  may 
require  a  separate  analysis  of  each  subject,  we  are 
concerned  with  the  issue  of  inter-subj  tct  variabil¬ 
ity.  The  variability  of  ERC  patterns  between  sub¬ 
jects  for  the  pre-stimulus  interval  of  the  present 
experiment  has  been  discussed  elsewhere  (Gevins 
et  al.  1987)  and  will  be  summarized  in  part  II.  We 
are  currently  investigating  the  inter-subject  vari¬ 
ability  of  the  other  events  (Gevins  et  al.  in  prep.). 

The  measurement  of  event-related  covariances 
is  currently  the  only  practical  means  of  idenufying 
narrow-band,  fraction-of-a-second  patterns  of 
wave  shape  similarity  and  timing  differences  be¬ 
tween  event-related  signals  recorded  from  differ¬ 
ent  scalp  sites.  Measurements  of  spectral  coher¬ 
ence,  as  are  conventionally  estimated  from 
smoothed  periodograms  or  autoregressive  models, 
have  not  yet  achieved  a  similar  degree  of  temporal 
and  frequency  resolution  (reviewed  in  Gersch 
1987).  Likewise,  measures  of  mutual  information, 
used  to  esumate  the  degree  of  relatedness  between 
2  averaged  event-related  time  series  (Mars  and 
Lopes  da  Silva  1987),  require  a  substantially  longer 
data  segment.  By  definition,  topographic  maps  of 
potential  or  metabolic  activity  do  not  show  this 
type  of  information. 

Conclusion 

Our  findings  suggest  that  a  network  modei. 
‘connecuonist.'  approach  to  human  brain  function 
is  feasible.  There  are  indications,  from  the  stimu¬ 
lus-locked  ERC  intervals,  for  example,  that  corti¬ 
cal  processing  of  the  meaning  of  the  stimulus 
evolves  into  processing  the  choice  of  response  at 
approximately  300  msec  post  stimulus.  This  find¬ 
ing  is  consistent  with  models  of  cortical  informa¬ 
tion  processing  (John  and  Schwartz  1978)  that 
distinguish  between  early  exogenous  and  later  en- 
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dogenous  processes.  The  response-related  ERC 
patterns  agree  with  multi-component  neurophysio¬ 
logical  and  radiological  models  of  the  cortical 
activity  that  accompanies  voluntary  responses 
(Roland  et  al.  1982;  Deecke  et  al.  1985).  Taken 
together,  the  results  presented  here  clearly  suggest 
that  intelligible  signs  of  task-specific  coordination 
are  available  from  the  distributed  mass  neural 
processes  of  the  brain  (Freeman  1975,  1987).  In 
part  II  of  this  report,  we  present  the  application  of 
the  ERC  method  to  higher  cognitive  functions. 
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Event-related  covariances  during  a  bimanual  visuomotor  task. 
II.  Preparation  and  feedback  1 
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Summary  Event-related  covariance  (ERC)  patterns  were  computed  from  pre-stimulus  and  feedback  intervals  of  a  bimanual, 

visuomotor  judgment  task  performed  by  7  nght-handed  men.  Late  contingent  negative  variation  (CNV)  ERC  patterns  that  preceded 
subsequently  accurate  right-  or  left-hand  responses  differed  from  patterns  that  preceded  subsequently  inaccurate  responses. 
Recordings  from  electrodes  placed  at  left  frontal,  midline  antero-central.  and  appropriately  contralateral  central  and  parietal  sites 
were  prominent  in  ERC  patterns  of  subsequently  accurate  performances.  This  suggests  that  a  distributed  cortical  'preparatory 
network,'  composed  of  distinct  cognitive,  integrative  motor,  somesthelic.  and  motor  components,  is  essential  for  accurate  visuomotor 
performance. 

ERC  patterns  related  to  feedback  about  accurate  and  inaccurate  responses  were  similar  to  each  other  in  the  interval  immediately 
after  feedback  onset,  but  began  to  differ  in  an  interval  spanmng  an  early  P300  peak.  The  difference  became  even  greater  in  an 
interval  spanning  a  late  P300  peak.  For  both  early  and  late  P300  peaks,  ERC  patterns  following  feedback  about  inaccurate 
performance  involved  more  frontal  sites  than  did  those  following  feedback  about  accurate  performance. 

Together  with  the  stimulus-  and  response-locked  results  presented  in  part  I.  results  of  this  study  on  the  preparatory  and  feedback 
periods  suggest  that  ERCs  show  salient  features  of  the  rapidly  shifting,  functional  cortical  networks  that  are  responsible  for  simple 
cognitive  tasks.  ERCs  thus  provide  a  new  perspective  on  information  processing  in  the  human  brain  in  relation  to  behavior  —  a 
perspective  that  supplements  conventional  EEG  and  ERP  procedures. 

Key  words:  Event-related  covariances;  Event-related  potentials;  Contingent  negative  variation;  P300;  Spatiotemporal  mapping; 

Preparatory  networks;  Feedback  networks;  Performance  accuracy 


As  a  step  towards  characterizing  the  distributed 
functional  cortical  networks  that  underlv  higher 
cognitive  functions,  we  have  been  measuring  the 
spatial  synchronization  of  event-related  potentials 


1  A  part  of  the  preparation  results  reported  here  have  ap¬ 
peared  in  abbreviated  form  in  Gevins  et  al.  1987.  In  addition 
to  a  more  complete  description  of  cue-locked  ERPs,  this  paper 
presents  the  ERPs  and  ERCs  for  the  feedback-locked  event, 
and  a  discussion  relating  the  contingent  negative  vanation 
(CNVV  and  P300-interval  results  to  each  other  and  to  other 
psychophystological  research. 

Correspondence  to:  A  S.  Gevins,  EEG  Systems  Laboratory. 
51  Federal  Street,  San  Francisco,  CA  94107  (USA). 


(ERPs)  (Gevins  et  al.  1981.  1983,  1985,  1987).  In 
this  experiment,  we  measured  the  spatial  synchro¬ 
nization  of  ERPs  related  to  preparation,  stimulus 
processing,  response  execution  and  feedback  about 
response  accuracy  during  a  4  sec  long  bimanual 
visuomotor  task.  The  results  reported  in  part  I 
(Gevins  et  al.  1989)  showed  that  event-related 
covariance  (ERC)  patterns  of  stimulus  processing 
and  response  execution  correspond  to  existing 
models  of  the  spatial  and  temporal  organization  of 
cortical  function.  The  results  presented  in  this 
second  report  show  that  the  ERCs  of  pre-stimulus 
and  feedback  intervals  are  related  to  the  accuracy 
of  performance  and  are  also  consistent  with  recog¬ 
nized  neurocognitive  models.  ERCs  provide  a  new 
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means  of  characterizing  performance-related 
aspects  of  neural  processing  and  thereby  open  new 
avenues  for  understanding  the  CNV  and  P300 
waves. 


Methods 

Subjects,  task  and  recordings 

Detailed  methods  of  this  study  were  presented 
in  part  I  and  will  be  reviewed  briefly  here. 

EEGs  were  recorded  from  7  right-handed, 
healthy  male  adults  while  they  performed  a  well- 
practiced  visuomotor  task.  Each  trial  of  the  task 
began  with  a  cue  symbol  (‘V’),  which  was  slanted 
to  the  right  or  left  to  indicate  with  which  hand  the 
subject  should  respond.  One  second  after  this  cue. 
a  visual  stimulus  (no.  1-9)  appeared.  In  "response" 
trials,  the  stimulus  was  slanted  in  the  same  direc¬ 
tion  as  the  cue.  The  subject  had  to  respond  quickly 
with  the  index  finger  of  the  indicated  hand,  by 
producing  a  pressure  from  0.1  to  0.9  kg  linearly 
related  to  the  stimulus  number.  In  a  randomly 
distributed  20'?  of  the  trials,  the  stimulus  was 
slanted  in  the  direction  opposite  to  the  cue.  In 
these  ‘  no-response"  catch  trials,  the  subject  had  to 
withhold  the  finger  response. 

Feedback  was  presented  visually  1  sec  after  the 
subject's  peak  response  pressure  in  response  trials. 
Feedback  indicated  the  actual  pressure  (to  a  tenth 
of  a  stimulus  unit)  the  subject  made.  If  the  re¬ 
sponse  was  sufficiently  close  to  the  required  re¬ 
sponse  (according  to  a  continuously  updated  toler¬ 
ance  of  adaptive  error),  the  feedback  number  was 
underlined  to  indicate  a  ‘win.’  The  purpose  of  the 
adaptive  error  tolerance  was  to  adjust  the  margin 
for  winning  as  performance  fluctuated.  The  error 
tolerance  was  computed  for  each  hand  separately 
as  a  function  of  performance  during  the  previous 
5  trials.  Subjects  earned  a  bonus  of  5  cents  for 
each  win  and  were  penalized  10  cents  for  each 
response  to  a  ‘no-response’  catch  trial.  Summaries 
of  performance  and  the  amount  of  money  earned 
were  presented  at  the  end  of  each  block  of  17 
trials.  Together  with  frequent  rest  breaks,  these 
procedures  were  intended  to  help  subjects  remain 
alert  during  the  5-7  h  recording  session. 


Using  visual  editing,  we  discarded  trials  that 
contained  physiological  or  instrumental  artifact, 
slow,  hesitant,  or  delayed  responses,  or  any  sign  of 
EMG  in  the  non-responding  hand  or  in  the  cue- 
to-stimulus  epoch  in  the  responding  hand. 

Performance  accuracy 

In  order  to  identify  neuroelectric  patterns  asso¬ 
ciated  with  preparation  and  quality  of  perfor¬ 
mance.  each  subject’s  trials  were  sorted  according 
to  hand  and  response  accuracy.  ‘Accurate"  and 
‘inaccurate"  data  sets  contained  trials  in  which  the 
response  error  (deviation  from  required  finger 
pressure)  was  less  or  greater,  respectively,  than  the 
mean  error  calculated  over  each  subject's  •mtire 
recording  session.  Accurate  and  inaccurate  trials 
were  distributed  uniformly  across  the  session  and. 
therefore,  did  not  differ  because  of  learning  or 
possible  systematic  shifts  in  arousal  across  the 
session. 

Trials  were  also  sorted  according  to  hand  and 
accuracy  during  feedback-locked  intervals.  Behav¬ 
ioral  variables  such  as  response  time,  pressure, 
and  duration  were  carefully  balanced  to  eliminate 
the  possibility  that  differences  in  post-response 
activity,  unrelated  to  our  neurophysiological  hy¬ 
potheses.  might  extend  to  the  onset  of  feedback. 

Laplactan  derivation  wave  forms  and  event-related 
covariances 

Sixteen  laplacian  derivation  (LD)  channels  were 
computed  from  recordings  by  a  total  of  26  scalp 
electrodes.  The  16  were  the  non-peripheral  scalp 
electrodes  with  adequate  sampling  of  surrounding 
potential.  Enhanced,  averaged,  event-related.  LD 
time  series  were  formed  from  sets  of  trials  that 
contained  detectable  event-related  signals  (Gevins 
et  al.  1986).  A  positive  peak  in  the  LD  wave  form 
represented  the  emerging  current  at  the  scalp  and 
is  referred  to  as  a  ‘current  source.'  (This  terminol¬ 
ogy  should  not  be  confused  with  an  actual  neural 
source  or  ‘generator.’)  Since  most  LD  peaks 
changed  polarity  across  the  scalp,  all  peaks  were 
labeled  according  to  standard  ERP  terminology 
(e.g..  CNV.  P300). 

ERCs  for  each  of  the  120  pairwise  combina¬ 
tions  of  the  16  non-penpherai  channels  were  com¬ 
puted  from  the  enhanced,  filtered.  7-person  aver- 
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aged  LD  wave  forms  (see  part  I).  A  delta  (0.1-3 
Hz)  bandpass  filter  and  a  covariance  interval  width 
of  375  msec  were  used  to  emphasize  the  low- 
frequency.  late  post-cue  CNV  component  of  the 
LD  wave  form.  A  theta  (4-7  Hz)  bandpass  filter 
and  a  187  msec  wide  covariance  interval  were  used 
to  analyze  the  post-feedback  LD  components. 
Measuring  the  covariance  between  narrow  band¬ 
pass-filtered  wave  forms  was  equivalent,  in  princi¬ 
ple.  to  computing  the  magnitude  of  the  cross-spec¬ 
trum  in  the  frequency  domain.  However,  working 
in  the  time  domain  made  it  easier  to  derive  the 
optimal  trade-offs  between  time  and  frequency 
resolution  in  our  data.  Working  in  the  time  do¬ 
main  also  made  computations  efficient,  since  we 
could  calculate  only  over  the  band  of  interest,  and 
not  over  the  entire  spectrum. 

All  ERCs  with  significance  of  P  <  0.05  were 
displayed  for  pre-stimulus  CNV  intervals.  Time 
delays  could  not  be  computed  for  the  CNV.  be¬ 
cause  an  interval  wide  enough  to  allow  an  ade¬ 
quate  estimate  of  the  delay  would  have  included 
components  related  to  processing  of  the  cue  and 
would  thus  have  not  related  solely  to  the  CNV. 
ERCs  within  2  S.D.s  of  the  maximum  ERC  value 
were  displayed  for  the  feedback  intervals,  because 
of  the  many  highly  significant  ERCs.  As  described 
in  part  I.  ERC  patterns  were  compared  for  dif¬ 
ferences  in  magnitude  by  the  Student’s  i  test,  and 
for  differences  in  pattern  by  the  ‘bootstrap’  corre¬ 
lation  method  (Efron  1982). 

We  used  pattern  classification  procedures  to 
determine  how  well  the  pre-stimulus  ERCs  could 
predict  the  subsequent  performance  accuracy,  and 
to  assess  differences  between  subjects.  A  non-lin¬ 
ear.  2-layered  'neural  network'  pattern  classifica¬ 
tion  algorithm  (Viglione  1970;  Gevins  1980)  clas¬ 
sified  each  trial  for  subsequent  performance  accu¬ 
racy  by  forming  weighted  combinations  of  the 
pre-stimulus  ERCs.  Classification  equations  that 
consisted  of  weighted  combinations  of  the  deci¬ 
sions  of  discriminant  functions  were  computed 
using  a  recursive  procedure.  The  discriminant 
functions  themselves  consisted  of  weighted  combi¬ 
nations  of  a  subset  of  the  ERCs  shown  in  Fig.  2. 
We  validated  this  procedure  by  classifying  the 
trials  of  each  subject  by  equations  developed  on 
the  trials  of  the  other  6  subjects.  We  used  a 
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binomial  distribution  to  determine  the  significance 
of  the  average  of  the  7  validations.  A  more  de¬ 
tailed  discussion  of  the  application  of  pattern  clas¬ 
sification  procedures  to  neuroelectric  signals  can 
be  found  in  Gevins  (1980.  1987)  and  Gevins  and 
Morgan  (1986.  1988). 

Results 

Behavioral  analysis 

For  accurate  trials,  the  mean  deviation  of  re¬ 
sponse  pressure  from  the  required  pressure  was 
0.035  x  0.020  kg  for  the  right  hand,  and  0.039  r 
0.020  kg  for  the  left  hand.  The  comparable  values 
for  inaccurate  performance  were  0.162  +  0.066  kg 
for  the  right  hand  and  0.166  x  0.076  kg  for  the  left 
hand.  Mean  reaction  times  for  the  4  performance 
conditions  in  the  final  data  sets  ranged  con¬ 
sistently  from  610  to  618  msec. 

Cue -locked  wave  forms 

Although  visual  inspection  of  the  pre-stimulus 
CNV  in  the  7-person  averaged,  delta-filtered  LD 
wave  forms  (Fig.  1)  suggested  differences  in  am¬ 
plitude  that  were  related  to  condition,  such  dif¬ 
ferences  were  not  statistically  significant.  The 
mean  squared  amplitude  (over  channels)  of  the 
CNV  wave  form,  thought  to  be  related  to  prepara¬ 
tory  set  (Walter  1967;  reviewed  in  Tecce  1972). 
was  measured  for  375  msec  wide  intervals  (corre¬ 
sponding  to  covariance  analysis  intervals)  under 
subsequently  accurate  and  inaccurate,  right-  and 
left-hand  conditions.  In  the  interval  centered  at 
687  msec  post  cue  (covering  the  late  peak  of  the 
CNV).  the  amplitudes  cf  the  CNV  wave  form  for 
subsequently  accurate  and  inaccurate  perfor¬ 
mances  did  not  differ  significantly  for  either  hand. 
The  similarity  of  the  late  CNV  amplitudes  was 
also  confirmed  by  the  bootstrap  correlation  proce¬ 
dure.  When  we  compared  the  distributions  of  am¬ 
plitude  of  the  687  msec  interval  of  subsequently 
accurate  and  inaccurate  trials,  the  correlations  were 
0.84  +  0.16  for  right-hand  trials  and  0.83  ±0.14 
for  left-hand  trials.  Thus,  the  late  CNVs  under 
conditions  of  accurate  and  inaccurate  perfor¬ 
mance  did  not  differ  either  in  magnitude  or  in 
spatial  pattern. 


150 


A  S.  GEVINS  ET  AL. 


CUE  TO  PREPARE 
FOR  AN 

INDEX  FINGER  RESPONSE 


LEFT  HANO 


RIGHT  HAND 


ACCURATE  INACCURATE 
TRIALS  TRIALS 


ACCURATE  INACCURATE 
TRIALS  TRIALS 


A 


B 


C 


D 


-1  O  SEC  0  -I  0  SE  C 


CUE  STIM  CUE  ; 


Fr  I  I Fi  I  ' 


aCz 

i aCz  !  1 

1  Cl  || 

|Cl 

XO 

1  ■"  T: 

1  1  • 

i  i  ’ 

JUi 

’  !  ! 

i  1 1 

J^aOz_^ 

!  1  i 

1  . 

-1 - A 

I  0  SEC  0  -10  SEC  0 

CUE  STIM  CUE  STIM 


Fig.  1.  LaplacGn  derivation  (LD)  wave  forms  from  midline 
electrode  Fi.es,  showing  the  contingent  negative  variation 
(CNV),  .iid  the  event-related  covariance  (ERC)  analysis  inter¬ 
val  centered  at  687  msec  post-cue  in  the  cue-to-stimulus  period. 
Th-  mean  late  CNV  amplitudes,  from  low-pass  filtered  (below 
7  Hz)  averages  of  7  subjects,  are  not  significantly  different  for 
the  comparison  of  left-accurate  (A)  with  left-inaccurate  (B) 
conditions,  or  right-accurate  (C)  with  right-inaccurate  (D)  con¬ 
ditions. 


In  contrast  to  the  non-specific  CNV  amplitude 
patterns  that  were  seen  across  all  LD  channels, 
there  were  differences  in  the  CNV  amplitude  of 
the  left  and  right  antero-central  (motor-related) 


TABLE  I 

Mean  absolute  amplitude  of  the  laplacian  derivation  wave 
form  from  a  375  msec  wide  interval  centered  at  750  msec  post 
cue  on  the  late  CNV.  The  left  and  right  antero-central  sites 
(aC3  and  aC4)  are  thought  to  overly  primary  motor  areas.  The 
antero-central  CNV  was  larger  contralateral  to  the  hand  indi¬ 
cated  by  the  slant  of  the  cue  preceding  accurate  and  inaccurate 
performance.  The  asymmetries  that  preceded  inaccurate  per¬ 
formance  tended  to  be  larger.  Values  are  expressed  in  units  of 
jiV/cm:. 


Electrodes 

Accurate 

Inaccurate 

Right  cue 

Left  cue 

Right  cue 

Left  cue 

aC3 

0.63 

0.15 

0.83 

0.14 

aC4 

0.18 

0.36 

0.14 

0.50 

sites.  The  results  summarized  in  Table  I  show  that 
the  absolute  amplitude  of  the  late  CNV.  for  the 
375  msec  wide  interval  centered  at  750  msec  post 
cue.  was  appropriately  lateralized  to  the  hemi¬ 
sphere  contralateral  to  the  corresponding  hand  for 
both  accurate  and  inaccurate  performance.  For 
the  right  hand,  the  left  antero-central  amplitude 
was  greater  than  the  right  by  more  than  4:1.  For 
the  left  hand,  the  right  antero-central  was  greater 
than  the  left  by  2 : 1  or  more.  If  CNV  amplitude 
lateralization  is  taken  as  a  sign  of  response  prep¬ 
aration.  it  is  surprising  that  the  asymmetries  pre¬ 
ceding  inaccurate  performance  tended  to  be  larger. 

Cue-locked  ERCs 

ERCs  were  computed  from  delta  band-filtered, 
averaged  LD  wave  forms  in  each  of  several  inter¬ 
vals  during  the  500  msec  preceding  the  stimulus. 
Visually  distinct  and  statistically  reliable  perfor¬ 
mance-related  differences  were  first  clearly  ap¬ 
parent  in  the  interval  centered  687  msec  post  cue. 


Fig.  2.  View  of  the  significant  ( P  <  0.05)  late  CNV  ERC  patterns  (colored  lines),  superimposed  on  maps  of  late  CNV  amplitude. 
Both  ERCs  and  CNV  amplitude  measurements  are  from  an  interval  500-875  msec  after  the  cue  for  subsequently  accurate  and 
inaccurate  right-hand  (A.B)  and  left-hand  (C.D)  visuomotor  task  performance  by  7  right-handed  subjects.  The  thickness  of  an  ERC 
line  is  proportional  to  its  significance  (from  0.05  to  0.005).  A  violet  line  indicates  that  the  ERC  is  positive,  while  a  blue  line  indicates 
that  the  ERC  is  negative.  Because  of  the  short  ERC  analysis  interval,  time  delays  could  not  be  computed  for  the  CNV.  The  color 
scale  at  the  left  of  each  picture  represents  wave  amplitude  and  covers  the  range  from  the  minimal  to  maximal  values  of  the  2  maps. 
ERCs  involving  left  frontal  and  appropriately  contralateral  central  and  parietal  electrode  sites  are  prominent  in  patterns  for 
subsequently  accurate  performance  by  both  hands.  The  magnitude  and  number  of  ERCs  that  precede  subsequenilv  inaccurate 
left-hand  performance  are  greater  and  are  more  widely  distributed  compared  to  the  accurate  pattern.  For  the  right  hand,  fewer  and 
weaker  ERCs  characterize  subsequently  inaccurate  performance.  The  amplitude  maps  are  very  similar  for  the  4  conditions  and  do  not 

indicate  any  of  the  specific  differences  evident  in  the  ERC  patterns 
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i.e.,  313  msec  pro  stimulus  (Fig.  2).  These  ERC 
patterns  were  quite  distinct  from  those  related  to 
overt  finger  response  pressures  (part  I). 

During  the  period  between  the  cue  and  the 
stimulus,  the  averaged  event-related  muscle  poten¬ 
tial  (right  and  left  EMG)  and  eye  movement 
(vertical  and  horizontal  EOG)  channels  were  at 
the  noise  level  and  did  not  differ  between  condi¬ 
tions.  These  low  levels  confirmed  that  the  ERCs 
were  neural  in  origin  and  were  not  associated  with 
overt  movements  (Fig.  3). 

In  the  687  msec  centered  interval.  ERCs  associ¬ 
ated  with  subsequently  accurate  right-hand  perfor¬ 
mance  involved  sites  primarily  over  the  left  hemi¬ 
sphere  (Fig.  2A).  All  24  significant  ERCs  involved 
left-sided  sites  and  18  (75%)  of  these  were  exclu¬ 
sively  left-sided.  The  most  significant  ERCs  in¬ 
volved  left  frontal,  central,  and  parietal  sites. 

The  687  msec  centered  pattern  of  subsequently 
accurate  left-hand  performance  involved  a  greater 
proportion  of  right  hemisphere  sites  than  did  the 
accurate  right-hand  pattern  (Fig.  2C).  Of  18  sig¬ 
nificant  ERCs.  13  (72%)  involved  electrode  sites 
over  the  right  hemisphere.  There  was  only  1  left¬ 
sided  intrahemispheric  covariance,  between  aPl 
and  F3. 

During  this  same  interval,  there  were  only  2 
significant  ERCs  associated  with  subsequently  in¬ 


accurate.  right-hand  performance,  namely  left 
parietal  and  antero-panetal  to  left  frontal  (Fig. 
2 B >.  ERCs  for  subsequently  inaccurate  left-hand 
performance  (Fig.  2D)  were  more  bilaterally  sym¬ 
metric  and  complex  than  those  for  subsequently 
accurate  left-hand  performance. 

For  right-hand  performances,  the  mean  magni¬ 
tude  of  event-related  covariance  in  the  687  msec- 
centered  interval  was  significantly  larger  when 
performance  was  subsequently  accurate  (4.86  r 
0.34)  than  when  it  was  subsequently  inaccurate 
(3.90  ±0.41)  (Student's  r  =  7.7.  Jf  =  23.  P  <  5  x 
10  !<).  By  contrast,  the  mean  ERC  preceding  ac¬ 
curate  left-hand  performance  (4.10  ±0.68)  was 
smaller  than  that  preceding  inaccurate  left-hand 
performance  (4.93  ±  0.73)  ( t  =  5.6.  ilf  -  38.  P  <  5 
x  10  h). 

We  calculated  the  bootstrap  correlation  be¬ 
tween  the  distributions  of  significant  ERCs  of 
accurate  or  inaccurate  conditions,  to  assess  the 
similarity  of  ERC  patterns  irrespective  of  the  dif¬ 
ference  in  magnitude.  Right-hand,  subsequently 
accurate  and  inaccurate  ERC  patterns  were  corre¬ 
lated  at  0.57  ±  0.09.  while  the  correlation  for  the 
left  hand  was  0.10  ±0.14.  Thus,  accurate  and  in¬ 
accurate  preparatory  patterns  differed  in  both 
magnitude  and  pattern  for  both  hands.  The  right- 
hand  correlation  was  quite  high  (0.57).  given  that 
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Fig.  3  Averaged  vertical  and  horizontal  eve  movement  signals 
during  the  period  between  the  cue  and  the  stimulus  for  right- 
and  left-hand  trials  from  7  subjects  (A).  Averaged  muscle 
potential  signals  from  right  flexor  digitori  during  the  cue-to- 
siimuliks  epv>cb  iBv.  Muscle  potentials  during  overt  movements 
were  manv  times  larger  (O.  Averaged  muscle  potential  signals 
from  left  flexor  digitori  showed  the  same  relation.  Since  there 
is  no  evidence  of  eve  or  muscle  movements  during  preparation 
of  either  the  right-  or  left-hand,  the  F.RC  patterns  in  Fig.  2  are 
not  the  result  of  overt  movements. 


there  were  only  2  significant  ERCs  in  the  right 
inaccurate  pattern  (Fig.  2B).  This  indicates  that 
the  ordering  of  the  non-significant  F.RC's  in  that 
pattern  was  roughly  similar  to  that  of  the  accurate 
pattern,  and  that  the  difference  in  magnitude  was 
the  overriding  effect. 

We  used  statistical  pattern  classification  of 
F.RC's  during  the  interval  between  the  cue  and  the 
stimulus  to  assess  how  well  the  ERCs  dis¬ 
criminated  subsequent  performance  on  a  trial-bv- 
trial  basis,  and  to  measure  differences  between 
subjects.  These  results  have  been  described 
elsewhere  (Gevins  el  al.  19, S7)  and  are  summarized 
here. 

FRCs  from  6  of  (he  subjects  were  used  to 
discriminate  the  subsequent  accuracy  of  the  re¬ 
maining  subject  s  performance.  This  was  repeated 
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7  times.  The  FRCs  shown  in  Fig.  2  were  the 
variables  that  we  submitted  to  these  discrimina¬ 
tions.  Using  this  leave-out-one-person  validation, 
both  left-  and  right-hand  overall  classifications 
(577  and  597  respectively)  were  significant  at 
P  <  0.01.  Six  of  the  7  subjects  had  similar  patterns 
for  right-hand  discriminations,  while  there  were  2 
distinct  groups  for  left-hand  discriminations.  For 
the  subject  with  the  most  trials,  the  single-subject 
classification  was  687  (  P  <  0.001 )  for  subsequent 
right-  and  627  t  P  <  0.01 )  for  subsequent  left-hand 
performance  (as  determined  by  5  leave-oul-one- 
fil'th  replications).  Thus,  the  subjects'  preparatory 
patterns  for  both  accurate  and  inaccurate  right- 
hand  performances  were  verv  similar,  while  their 
left-hand  patterns  distinguished  the  subjects  into  2 
groups.  More  detailed  analysis  of  individual  sub- 
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Fig  4  Feedback -evoked  I  I)  wave  forms  al  rmdhne  sites  for 
left-hand  accurate  and  inaccurate  performance  trial v  showing 
eurlv  and  later,  more  long-lasting.  P3  waves.  I  he  earls  P3 
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ject  data  will  be  presemed  elsewhere  (Gevins  et  al. 
in  prep.). 

Feedback-locked  wave  forms 

The  7-person  averaged,  unfiltered  LD  wave 
forms  (Fig.  4)  contained  early  peak*  before  200 
msec  after  the  feedback.  They  corresponded  to  N1 
and  P2  and  were  similar  to  those  seen  in  the 
stimulus  event  (see  part  I).  A  small  peak,  corre¬ 
sponding  to  the  N2  wave,  was  distinguishable  at 
about  250  msec,  with  entering  current  at  the  mid¬ 
line  antero-occipital  site  and  exiting  current  at  the 
lateral  parietal  sites  (not  shown). 

Two  closely  overlapping  late  LD  peaks,  corre¬ 
sponding  to  positive  potentials,  had  differing 
topographies  in  the  different  experimental  condi¬ 
tions.  Theta  band  filtering  accentuated  the  condi¬ 
tion-specific  differences  of  these  wave  forms  bv 
removing  the  delta  band  portion  of  the  wave  form. 
Table  II  shows  the  sites  of  the  maxima  for  current 
sources  and  sinks  of  the  positive  peaks  during 
feedback  about  accurate  and  inaccurate  right-  and 
left-hand  performances. 

There  was  a  robust  early  ‘P3E’  peak  at  281-328 
msec  during  feedback  about  accurate  perfor¬ 
mance,  with  emerging  current  at  all  midline  sites 
and  lateral  parietal  sites  (maximal  at  the  midline 
antero-central  site),  and  entering  current  at  lateral 
frontal  sites  (maximal  at  the  right  frontal  site). 
The  later,  wider  'P3L'  peak  at  430-500  msec  had 
its  maximal  current  source  at  the  midline  antero- 
parietal  site,  and  its  maximal  sink  at  the  right 
frontal  site.  The  amplitude  of  the  P3E  peak  was 
larger  than  that  of  the  later  P3L. 


Feedback  about  inaccurate  performance  w'as 
also  characterized  by  2  peaks  that  occurred  be¬ 
tween  300  and  600  msec.  The  P3E  peak,  at  305-328 
msec,  w'as  similar  in  topography  to  that  of  the 
accurate  performance,  but  had  a  lower  overall 
amplitude.  The  later  P3L,  at  460-600  msec,  was 
larger  than  that  of  the  accurate  performance.  The 
site  of  maximal  exiting  current  for  the  P3L  was 
more  anterior,  at  the  midline  antero-central  site, 
and  its  amplitude  was  larger  than  the  amplitude  of 
the  P3E. 

For  both  accurate  and  inaccurate  performance, 
peak  amplitudes  at  500  msec  during  feedback 
about  right-hand  performance  were  larger  at  the 
left  '"cntral  and  left  antero-central  sites  than  at 
corresponding  sites  of  the  nght  hemisphere.  Simi¬ 
larly.  amplitudes  at  500  msec  for  feedback  about 
left-hand  performance  were  larger  at  the  right 
antero-central  site  for  accurate  performance,  and 
larger  at  the  right  central  site  for  inaccurate  per¬ 
formance. 

Feedback-locked  ERCs 

In  the  interval  spanning  the  early  ‘exogenous- 
N1  and  P2  feedback  peaks.  ERCs  from  theta 
band-filtered,  averaged  LD  wave  forms  did  not 
differ  according  to  accuracy  of  each  hand  and 
resembled  the  early  stimulus-locked  patterns 
shown  in  part  I. 

Differences  between  accurate  and  inaccurate 
conditions  began  to  emerge  in  the  interval  centered 
at  281  msec  that  spanned  the  early  P3  peak  (P3E). 
When  the  ERC  patterns  for  feedback  to  accurate 
and  inaccurate  performance  were  compared,  the 


Fig.  5.  Most  significant  (top  2  S.D.s;  within  2  S.D.s  of  largest  value)  early  P3  (P3E)  feedback  ERC  patterns  for  feedback  about 
accurate  and  inaccurate  performance  by  the  nght  hand  (A  and  B)  and  left  hand  (C  and  D).  ERC  patterns  are  supenmposed  on  color 
maps  of  wave  amplitude.  The  thickness  of  an  ERC  line  is  proportional  to  the  negative  log  of  its  significance  (from  0.05  to  0.00005). 
The  color  scale  at  the  left  of  each  picture  represents  wave  amplitude  and  covers  the  range  from  the  minimal  to  maximal  values  of  the 
2  maps.  Both  the  ERCs  and  amplitude  maps  are  derived  from  a  187  msec  wide  interval,  centered  at  281  msec  after  ihe  onset  of 
feedback,  on  theta  band-filtered  wave  forms  averaged  over  all  7  subjects  A  major  difference  between  accurate  and  inaccurate 
patterns  is  that  the  left  and  midline  frontal  sites  are  only  involved  in  the  inaccurate  patterns.  The  involvement  of  these  sues  may 
r-flect  the  fact  that  greater  processing  is  required  after  inaccurate  performance,  in  order  to  improve  subsequent  performance. 

Fig.  6.  Most  significant  (top  2  S.D  s)  late  P3  ERC  patterns  for  feedback  to  accurate  and  inaccurate  performance  by  ihe  nght  hand  (A 
and  B)  and  left  hand  (C  and  D).  Both  the  ERCs  and  amplitude  maps  are  derived  from  a  187  msec  wide  interval,  centered  at  468  msec 
after  feedback  onset,  on  theta  band-filtered  7-subjecl  averaged  wave  forms.  Involvement  of  the  left  panetal  site,  greater  involvement 
of  the  nght  panetal  site,  and  absence  of  the  nght  frontal  site  distinguished  the  accurate  patterns  from  the  inaccurate. 
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bootstrap  correlation  was  0.36  ±  0.35  for  right- 
and  0.73  t  0.05  for  left-hand  performance.  The 
ERC  patterns  for  feedback  about  accurate  perfor¬ 
mance  by  the  two  hands  (Fig.  5A  and  C)  were 
highly  similar  (bootstrap  correlation  =  0.91  ± 
0.01 ).  and  involved  midline  antero-central,  central, 
antero-parietal,  parietal  and  antero-occipital  sites, 
left  antero-parietal  and  antero-central  sites,  and 
right  parietal,  antero-panetal.  antero-central.  and 
frontal  sites.  These  accurate  patterns  involved 
many  long-delav  (32  79  msec)  ERCs.  The  wave 
forms  of  the  frontal  and  antero-central  sites  con¬ 
sistently  lagged  those  of  more  posterior  sites. 

The  ERC  patterns  during  feedback  about  inac¬ 
curate  performance  by  the  left  and  right  hands 
(Fig.  5B  and  D)  were  also  very  similar  (bootstrap 
correlation  =  0.90  ±  0.02).  They  involved  most  of 
the  same  sites  as  the  accurate  patterns,  with  the 
striking  exception  of  the  left  and  midline  frontal 
sites.  Again,  frontal  wave  forms  lagged  those  of 
the  more  posterior  sites  with  which  they  covaried. 
There  were  even  more  long-delay  ERCs  than  in 
the  accurate  patterns. 

In  the  interval  centered  at  468  msec,  spanning 
the  late  P3  peak  (P3L),  the  patterns  for  accurate 
and  inaccurate  performance  were  still  very  differ¬ 
ent  (bootstrap  correlation  -  -0.38  ±  0.27  for 
right-  and  0.13  ±  0.08  for  left-hand  performance). 
The  patterns  during  fee  (back  about  right-  and 
left-hand  accurate  performance  had  many  sites  in 
common  (e.g.,  midline  parietal,  antero-parietal. 
central,  antero-central.  and  frontal  sites,  bilateral 
parietal  sites,  and  left  antero-central  and  frontal 
sites),  although  they  were  less  similar  than  in  the 
early  P3  interval  (bootstrap  correlation  =  0.58  x 
0.06)  (Fig.  6A  and  C).  The  absence  of  the  right 
frontal  site  from  these  patterns  was  conspicuous, 
considering  its  prominence  in  the  accurate  pat¬ 
terns  from  the  early  P3  interval.  The  inaccurate 
patterns  were  also  less  similar  than  in  the  early  P3 
interval  (bootstrap  correlation  =  0.77  ±  0.04).  They 
differed  from  the  accurate  ones,  particularly  in 
that  the  right  frontal  site  was  involved,  the  left 
parietal  site  was  absent,  and  the  right  parietal  site 
was  less  involved  (Fig.  6B  and  D). 

Feedback  patterns  about  left-hand  accurate  and 
inaccurate  performance  had  more  ERCs  than  did 
patterns  during  feedback  about  right-hand  perfor- 
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Fig.  7.  Averaged  vertical  and  horizontal  eye  movement  signals 
from  7  .ubjects  during  feedback  about  accurate  and  inaccurate 
left-hand  performance. 

mance.  As  was  the  case  during  the  P3E  interval, 
the  similarity  of  P3L  ERC  patterns  between  the  2 
hands,  regardless  of  performance  accuracy,  was 
greater  than  between  accurate  and  inaccurate  pat¬ 
terns  for  either  hand.  This  implies  that  the  feed¬ 
back  ERC  patterns  in  both  intervals  were  related 
more  to  performance  accuracy  than  to  the  hand 
used. 

Vertical  and  horizontal  EOG  channels  were  at 
the  noise  level  for  all  conditions  of  accuracy  and 
handedness  (Fig.  7). 

Discussion 

Although  the  event-related  preparatory  and 
feedback  covariance  patterns  reported  here  appear 
to  represent  signs  of  ‘functional  coordination'  be¬ 
tween  nodes  within  a  cooperative  cortical  network, 
we  must  again  caution  that  such  an  interpretation 
is  speculative  until  the  generators  of  these  patterns 
are  determined.  With  this  in  mind,  we  offer  the 
following  comments  on  the  results  of  this  study. 

Preparation 

The  prominence  of  the  left  frontal  site  in  our 
preparatory  patterns  prior  to  actual  response  ex- 
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ecution  bv  either  hand  is  consistent  with  clinical 
evidence  that  prefrontal  cortical  areas  are  involved 
in  the  synthesis  of  preparatory  sets  (Luria  1966). 
and  that  the  left  dorsolateral  prefrontal  cortex  is 
specifically  involved  in  cued  delayed  response  tasks 
(Jacobsen  1935;  Pribram  et  al.  1977;  Shallice  1982; 
Fuster  1984).  Similarly,  the  prominence  of  the 
midline  central  and  antero-central  sites  suggests 
the  presence  of  an  integrative  motor  component. 
The  involvement  of  these  sites  in  the  preparatory 
patterns  is  consistent  with  clinical  evidence  for 
involvement  of  the  premotor  and  supplementary 
motor  regions  when  existing  motor  schemes  are 
initiated,  or  when  new  ones  are  established  (Pen- 
field  and  Jasper  1959;  Goldman- Rakic  1984; 
Milner  and  Petrides  1984;  Goldberg  1985;  Stuss 
and  Benson  1986).  The  integrative  motor  compo¬ 
nent.  strong  in  the  pattern  preceding  accurate 
left-hand  performance  and  weaker  in  the  pattern 
preceding  accurate  right-hand  performance,  may 
indicate  that  a  greater  degree  of  motor  control  is 
necessary  when  strongly  right-handed  subjects  ex¬ 
ecute  a  fine  motor  response  with  their  left  hands. 
The  existences  of  appropriately  lateralized  somes- 
thetic  and  motor  preparatory  components  are  sup¬ 
ported  by  experimental  evidence  for  distinct  firing 
patterns  of  neurons  in  the  motor  and  premotor 
cortices  preceding  motor  responses  in  non-human 
primates  (Evarts  et  al.  1984).  The  difference  be¬ 
tween  ERC  patterns  corresponding  to  preparation 
and  to  actual  response  execution  (see  part  I)  sug¬ 
gests  that  they  reflect  different  organizations  of 
neural  activity. 

The  detailed  spatial  organization  and  interelec¬ 
trode  covariance  of  the  late  CNV  clearly  reveal 
several  anatomically  and  functionally  distinct 
processes.  Although  traditional  CNV  studies  have 
tended  to  view  the  CNV  as  a  unitary  brain  process 
that  reflects  anticipation  of  a  response  based  on 
the  contingency  between  2  stimuli  (SI  and  S2) 
(Walter  1967;  reviewed  in  Tecce  1972).  there  has 
been  growing  evidence  that  the  CNV  is  a  multi¬ 
component  process  (Ruchkin  et  al.  1986;  Loveless 
et  al.  1987).  There  has  also  been  some  debate  as  to 
whether  the  CNV  reflects  only  motor  preparation, 
or  whether  there  is  also  a  cognitive  component 
related  to  the  expected  stimulus  (S2)  (Gaillard 
1977  (reviewed  in  Gevins  and  Cutillo  1986); 


Rohrbaugh  et  al.  1980;  Ruchkin  et  al.  1986).  As 
discussed  above,  the  present  results  support  the 
idea  that  preparation  is  a  multi-component  pro¬ 
cess  that  includes  not  only  motor-somesthetic 
preparation  but  also  cognitive  and  high-level  in¬ 
tegrative  motor  components.  Finally,  inconsistent 
results  of  previous  studies  that  related  the  CNV  to 
the  quality  of  subsequent  performance  could  be 
attributed,  at  least  in  part,  to  insufficient  spatial 
sampling  and  analysis  (for  recent  examples,  see 
Macar  and  Vitton  1982;  Macar  and  Besson  1986). 

Our  preparatory  findings  are  consistent  with 
‘continuous  flow'  models  of  the  temporal  organi¬ 
zation  of  human  cognition  (F.rikson  and  Schultz 
1979;  Coles  et  al.  1985).  The  temporal  evolution  of 
the  ERC  patterns  that  follow  the  vue  indicates 
that  response  preparation  begins  with  evaluation 
of  the  cue.  well  before  stimulus  presentation.  Like 
previous  workers  (Rohrbaugh  et  al.  1976;  Coles 
and  Gratton  1986).  we  found  a  slightly  larger 
CNV  amplitude  over  motor  areas  contralateral  to 
the  subsequently  responding  hand.  The  paper  by 
Coles  and  Gratton  demonstrated  that  the  response 
is  more  likely  to  be  made  with  the  correct  hand 
when  the  CNV  is  lateralized  coniralaterallv.  rather 
than  ipsilaterallv.  to  the  hand  indicated  by  the 
stimulus.  The  ERC  patterns  appear  to  be  more 
sensitive  to  preparatory  set  than  do  amplitude 
patterns;  thev  clearly  distinguished  subsequent 
near-target  (accurate)  from  far-from-target  (inac¬ 
curate)  index  finger  pressures,  both  of  which  were 
performed  with  the  correct  hand. 

Feedback 

The  ERC  patterns  of  both  the  early  and  late  P3 
peaks  in  the  feedback  interval  appeared  to  reflect 
a  difference  between  ‘confirming’  (following  accu¬ 
rate  performance)  and  ‘disconfirming'  (following 
inaccurate  performance)  feedback,  with  a  greater 
difference  for  the  later  peak.  In  the  early  interval, 
the  main  difference  between  the  confirming  and 
disconfirming  patterns  was  that  all  frontal  sites 
were  involved  in  disconfirming  feedback,  whereas 
the  left  and  midline  frontal  sites  were  absent  for 
confirming  feedback.  For  the  late  peak,  the  left 
and  midline  frontal  sites,  as  well  as  the  left  parietal 
site,  became  involved  in  the  pattern  for  confirming 
feedback,  and  the  right  frontal  site  was  no  longer 
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involved.  The  patterns  from  the  late  interval  for 
disconfirming  feedback  were  quite  different  from 
those  of  the  early  interval,  although  the  changes 
were  different  for  the  2  hands. 

ERC  patterns  for  P3  peaks  that  followed  dis¬ 
confirming  feedback  involved  more  frontal  sites 
than  did  patterns  that  followed  confirming  feed¬ 
back.  We  would  expect  a  greater  resetting  of  per¬ 
formance-related  neural  systems  following  discon¬ 
firming  feedback,  as  well  as  a  strong  frontal  in¬ 
volvement.  given  the  importance  of  the  frontal 
lobes  for  the  integration  of  sensory  and  motor 
activities  (Fuster  1984;  Stuss  and  Benson  1986). 
The  difference  between  right-  and  left-hand  pat¬ 
terns  was  greater  for  the  late  than  for  the  early 
peak,  with  both  left-hand  patterns  having  a  greater 
number  of  ERCs  than  the  right-hand  patterns. 
This  suggests  that  in  the  right-handed  subjects  of 
this  study,  resetting  these  systems  after  left-hand 
performance  required  more  processing  during  the 
late  peak. 

Prior  experiments  dealing  with  feedback-evoked 
potentials  have  shown  differences  in  P3  latency 
between  confirming  and  disconfirming  feedback 
to  auditory  stimuli,  with  confirming  feedback 
eliciting  a  short-latency  P3,  and  disconfirming 
feedback  eliciting  a  long-latency  P3  (Squires  et  al. 
1973).  The  latency  of  our  early  P3,  however,  did 
not  vary  with  feedback  conditions  as  it  did  in 
previous  studies,  and  our  late  P3  for  disconfirming 
feedback  was  delayed  only  about  30  msec.  Rather, 
the  late  P3  LD  peak  at  the  site  of  maximum 
exiting  current  was  larger  than  the  early  P3  peak 
during  feedback  about  inaccurate  responses,  but 
was  smaller  during  feedback  about  accurate  re¬ 
sponses  (Fig.  4).  The  major  difference  between 
confirming  and  disconfirming  feedback  wave 
forms  in  our  study,  therefore,  was  related  to  the 
relative  amplitudes  of  the  early  and  late  P3  peaks, 
rather  than  to  their  latency. 

The  P3  peaks  in  our  data  relate  to  late  positive 
waves  reported  in  other  studies  on  feedback  about 
accuracy.  From  studies  of  time  estimation.  John¬ 
son  and  Donchin  (1978.  1985)  concluded  that  the 
early  P3  is  associated  with  the  identification  and 
classification  of  the  feedback,  while  the  later  P3  is 
associated  with  the  utilization  of  feedback  infor¬ 
mation  in  the  reinforcement  or  revision  of  the 


response  (time  estimate).  In  other  studies,  particu¬ 
larly  those  in  which  monetary  bonuses  served  as 
an  incentive  for  good  performance  (e.g.,  Poon  et 
al.  1974;  Steinhauer  1981).  the  second  positive 
wave  was  attributed  in  part  to  additional  process¬ 
ing  (such  as  assessing  current  assets),  once  the 
nature  of  the  feedback  has  been  determined.  Like¬ 
wise,  Stuss  and  Picton  (1978)  suggested  that  serial 
processing,  i.e..  recognizing  the  meaning  of  a 
feedback  stimulus,  followed  by  using  that  infor¬ 
mation,  is  reflected  in  a  P3  component  followed 
by  a  long-latency,  broad  duration,  later  compo¬ 
nent  (which  they  termed  'P4').  The  increasing 
divergence  over  time  of  confirming  and  discon¬ 
firming  feedback  ERC  patterns  is  consistent  with 
the  general  conclusion  of  these  researchers  that 
the  second  late  positive  wave  is  related  to  utiliza¬ 
tion  of  information  extracted  from  the  feedback 
stimulus. 

Conclusion 

Using  modern  recording  and  signal  processing 
technologies,  we  have  attempted  to  improve  the 
spatial,  temporal,  and  neurocognitive  specificity  of 
earlier  studies.  In  the  spatial  domain,  we  increased 
the  number  of  channels  recorded  simultaneously 
and  reduced  volume  conduction  blurring.  Even 
with  these  advancements,  the  specificity  of  the 
results  suggests  that  a  further  increase  in  spatial 
sampling  will  be  beneficial.  In  the  temporal  do¬ 
main,  we  measured  shifts  on  the  order  of  tens  of 
milliseconds  between  event-related  peaks  of  differ¬ 
ent  channels.  Since  the  analysis  was  restricted  to 
specific  frequency  bands,  there  is  a  wealth  of  data 
in  other  frequency  regions  and  their  accompany¬ 
ing  time  intervals,  that  has  yet  to  be  explored. 

We  have  isolated  specific  cognitive  processes 
related  to  different  events  in  the  4  sec  trial,  using 
highly  controlled  tasks.  The  time  evolution  of 
ERC  patterns  over  the  course  of  the  whole  4  sec 
trial  should  provide  deeper  insight  into  the  neuro¬ 
cognitive  processing  involved.  In  light  of  the  stak¬ 
ing  findings  reported  here  and  in  part  I.  determin¬ 
ing  the  generators  of  scalp-recorded  event-related 
covariance  patterns  is  the  next  essential  step  in 
characterizing  the  dynamic  functional  cortical  net¬ 
works  that  underlv  cognition  and  behavior. 
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Abstrxut— This  paper  review*  the  application  o t  neural- network  (NN) 
ngnil  processing  methods  io  neurological  waveform  detection  and  pat¬ 
tern  analysis.  NN  methods  are  shown  to  be  an  excellent  way  of  incor¬ 
porating  expert  knowledge  about  the  brain  into  a  mathematical  frame¬ 
work  with  minimal  assumptions  a  boat  the  statistics  of  signals  and  noise. 
Constrained  by  expert  knowledge,  NN  algorithms  can  search  for  op¬ 
timal  or  near-optimal  connections  between,  and  weightings  of,  appli¬ 
cation-specific  features  in  data  spaces  for  which  human  knowledge  is 
Incomplete. 

Applying  NN  algorithms  to  electrical  signals  noninvasiveiy  recorded 
from  the  human  brain,  the  neurological  effects  of  different  types  of 
sleeping  pills  have  been  differentiated,  and  insights  have  been  gained 
as  to  how  our  brains  produce  higher  cognitive  functions.  Other  signal 
processing  problems  with  highly  dimensioned  noisy  data  may  be  ame¬ 
nable  to  similar  solutions.  However,  the  importance  of  application-spe¬ 
cific  constraints  in  solving  real-world  problems  should  not  be  under¬ 
estimated. 


Introduction 

XTRACTING  useful  information  from  neuroelectric 
data  recorded  at  the  scalp  requires  a  complex,  and 
often  unknown,  senes  of  processing  operations.  This  is  a 
multidimensional  signal  processing  problem  in  which  the 
signal  and  noise  have  unknown  but  similar  statistics,  in 
which  the  noise  generally  is  significantly  stronger  than  the 
signal,  and  in  which  the  signal  dimensions  are  smeared 
because  of  spatial  crosstalk. 

Because  of  the  difficulty  of  brain  signal  analysis,  con¬ 
ventional  open-loop  analyses  are  often  insufficiently  sen¬ 
sitive.  Pnor  selection  of  the  signal  features  either  solely 
by  "blind”  application  of  statistical  procedures  or  solely 
from  expert  opinion  may  not  differentiate  disease  pro¬ 
cesses  or  discriminate  the  conditions  of  an  experiment.  In 
the  former  case,  statistical  features  which  maximize  some 
criteria,  such  as  the  amount  of  variance  explained,  may 
be  irrelevant  for  distinguishing  clinical  categories  or  ex¬ 
perimental  conditions  whose  means  overlap.  In  contrast, 
while  expert  knowledge  is  invaluable  for  the  initial  choice 
of  appropriate  measurements  and  for  the  final  interpreta¬ 
tion  of  results,  features  chosen  by  experts  may  be  highly 

Manuscript  received  February  22,  1988.  This  work  waa  supported  by 
the  National  Institute  of  Neurological  and  Communicative  Diseases  and 
Stroke,  the  U  S.  Air  Force  Office  of  Scientific  Research,  and  the  Nauonai 
Science  Foundation. 

The  authors  are  with  the  EEG  Systems  Laboratory,  San  Francisco,  CA 
94103 

IEEE  Log  Number  8821367 


correlated  with  each  other  and  therefore  not  effective  in 
combination.  While  experts  may  have  a  good  intuitive 
grasp  of  multidimensional  patterns,  they  cannot  deter¬ 
mine  the  best  combinations  of  reatures  from  a  superset  of 
features  to  characterize  thousands  of  millions  of  data  sam¬ 
ples.  Furthermore,  the  brain’s  cTtual  comp»'2:^iiai  al¬ 
gorithms  underlying  experts’  decisions  (as  opposed  to  ex¬ 
perts'  English  language  descriptions  of  the  rules  they  use) 
are  not  often  available  for  inspection  since  they  are  coded 
in  the  “wetware"  of  the  brain. 

We  have  found  iterative  neural-network  (NN)  proce¬ 
dures  to  be  useful  in  overcoming  these  limitations  when 
analyzing  neuroelectric  signals  from  the  human  brain  [2]- 
[5].  Constrained  by  expert  knowledge,  these  techniques 
search  for  optimal  or  near-optimal  connections  of,  and 
weightings  between,  application-specific  features  in  data 
spaces  for  which  human  knowledge  is  incomplete  [18]. 
However,  in  our  experience,  it  is  not  useful  to  feed  large 
sets  of  data  into  a  neural  net  without  constraints  or  guid¬ 
ance  since  our  noisy,  multidimensional  data  require  the 
determination  of  combinations  of  a  relatively  small  num¬ 
ber  of  features  (feature  subsets)  in  order  to  generalize 
classification  past  the  training  data  set.  "Hidden  units” 
in  NN  explicitly  test  combinations  of  features  for  discrim¬ 
inative  power,  but  a  combinatorial  explosion  is  encoun¬ 
tered  when  exhaustively  selecting  a  small  number  of  fea¬ 
tures  from  a  larger  set.  A  fully  connected  Hopfield-type 
network  seemingly  obviates  the  need  for  feature  selec¬ 
tion,  but  practical  experience  with  noisy  and  time-varying 
experimental  data  suggests  that  resultant  classification 
networks  are  likely  to  be  far  from  optimal.  In  particular, 
Hopfield  networks  are  likely  to  settle  into  local  minima  in 
these  instances,  even  when  simulated  annealing  methods 
are  used.  Multilayer  perceptions  employing  gradient 
search  methods  are  prone  to  similar  difficulties. 

Our  approach  has  been  to  use  NN’s  to  select  combina¬ 
tions  of  application-specific  features  from  larger  sets  cho¬ 
sen  by  experts  (or  expert  systems)  [6]-[13] .  Big  prob¬ 
lems,  such  as  discriminating  the  brain  state  corresponding 
to  good  and  poor  performance,  are  divided  into  subprob¬ 
lems.  Small  candidate  sets  of  features  are  selected  for  the 
subproblems  by  reference  to  brain  anatomy  and  neuro¬ 
physiology.  NN’s  are  then  sequentially  applied  to  each  of 
the  feature  sets.  Network  topology  is  determined  by  prop¬ 
agation  of  classification  error  back  to  connections  and 
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Fig.  1  Neural  network  (NN)  for  feature  extraction  and  classification.  Per¬ 
formance  of  this  network  is  fed  back  to  guide  feature  selection. 


weighting  of  hidden  units  during  training  mode  operation 
[16],  [19].  To  generate  each  network,  the  algorithm  iter¬ 
ates  a  two-step  procedure  (Fig.  1).  First  a  "network  loss” 
is  calculated  for  a  two-layered  network  consisting  of  hid¬ 
den  threshold  logic  units  (TLU’s)  previously  chosen  and 
each  possible  new  hidden  unit.  Candidate  TLU’s  are  gen¬ 
erated  by  the  exhaustive  enumeration  of  all  feature  sub¬ 
sets  of  a  given  size  and  computing  a  simple  statistical 
classifier  for  each.  The  network  loss  is  a  function  defined 
to  reflect  both  the  quantity  and  severity  of  classification 
errors.  Error  correction  is  propagated  back  to  network  pa¬ 
rameters  by  minimizing  this  measure  of  the  difference  be¬ 
tween  desired  and  actual  output.  This  process  iterates  un¬ 
til  adding  more  units  does  not  result  in  significant 
improvements.  For  most  of  the  applications  described 
here,  several  hundred  patterns  are  used  to  select  a  final  set 
of  about  six  hidden  units  for  each  of  up  to  several  hundred 
subprobiems.  Independent  data  are  then  passed  through 
the  network  to  test  the  validity  of  the  final  network  equa¬ 
tions.  The  significance  of  this  test  is  determined  by  the 
binomial  distribution  and  by  comparison  to  results  ob¬ 
tained  when  data  of  random  class  are  passed  through  the 


network.  The  performance  of  each  network  is  fed  back  to 
guide  feature  selection.  Several  hundred  thousand  of  these 
network  configurations  are  trained  to  test  possible  feature 
subsets  for  each  subproblem  of  each  application.  The  final 
network  parameters  frequendy  yield  useful  information 
about  the  relevance  of  each  feature  for  classification. 

While  larger  networks  (such  as  a  three-layer  perception 
with  backward  propagation  and  a  sigmoid  nonlinearity) 
can  generate  more  complicated  decision  surfaces,  we  have 
found  the  critical  practical  problem  to  be  the  estimation 
of  network  parameters  from  noisy  data.  This  problem  is 
not  solved  by  a  more  complex  network,  but  can  instead 
be  treated  with  classifier-directed  feature  selection  meth¬ 
ods.  Evaluation  of  hundreds  of  thousands  of  simpler  clas¬ 
sifiers  results  in  small  feature  subsets  which  generalize 
well  to  new  data  in  the  presence  of  noise. 

Applications  of  Neural-Network  Signal 
Processing  to  Brain  Research 
Over  the  last  ten  years,  we  have  found  this  approach  to 
be  useful  in  our  multidimensional  signal  processing  work 
with  signals  of  unknown  characteristics  and  signai-to- 
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Fig.  2.  NN  artifact  (contaminant)  detection.  A  system  consisting  of  five 
parallel  detectors  is  used  to  find  contaminants 


noise  ratios  well  below  one.  We  now  review  some  of  these 
applications. 

A.  Waveform  Detection 

l)  Contaminant  Detection — Replacing  ad  hoc  Detec¬ 
tors:  The  characterization  of  infrequent,  morphologically 
variable  transient  events,  such  as  contaminants  arising 
from  eye  blinks,  eye  movement,  or  muscle  potentials,  is 
an  essential  component  of  neuroelectric  signal  analysis. 
Accordingly,  a  great  deal  of  energy  has  been  expended 
over  the  years  in  efforts  to  automatically  detect  these  phe¬ 
nomena,  but  with  mixed  results  (see  review  in  Barlow 
[1]). 

We  have  implemented  an  NN  system  for  automatic  re¬ 
jection  of  contaminants  of  neuroelectnc  signals  [5].  Ex¬ 
pert  knowledge  of  contaminant  types  is  represented  by 
training  data  which  have  been  hand  marked,  rather  than 
by  explicit  descriptions  of  the  features  a  human  uses  to 
detect  these  events.  These  hand-marked  data  are  used  to 
train  multilayered  networks  which  are  used  to  distinguish 
clean  from  contaminated  data. 

Parallel  layered-network  detectors  are  each  specialized 
for  a  different  type  of  contaminant  (Fig.  2).  Three  of  the 
detectors  incorporate  the  layered  NN  classifier  of  Fig.  1 
to  choose  and  weight  feature  combinations.  In  this  way. 


sole  reliance  on  ad  hoc  waveform  detectors  and  manually 
set  thresholds  is  replaced  by  a  precise  automated  proce¬ 
dure.  One  detector  uses  low-frequency,  neuroelectric  sig¬ 
nal  power  spectral  density,  cross-spectral  density  between 
neuroelectric  signals  and  horizontal  and  vertical  eye 
movement  signals,  and  correlation  coefficient  between 
neuroelectric  signals  and  vertical  eye  movement  signals 
as  features  to  detect  eye  blinks  and  eye-movement  con¬ 
tamination.  Another  layered- network  detector  uses  neu¬ 
roelectric  signal  power  spectral  density,  zero-lag  autocor¬ 
relation,  and  the  first-order  AR  predictor  coefficient  as 
candidates  for  muscle  potential  and  steady-state  high-fre¬ 
quency  instrumental  contamination.  An  optional  third  de¬ 
tector  uses  the  same  type  of  features  for  other  types  of 
contaminant  that  users  wish  to  train  the  machine  to  reject. 
Features  are  evaluated  over  half-second  intervals  every 
125  ms  of  the  time  series.  Classification  decisions  are 
made  every  125  ms  using  a  nonlinear  interpolation  of 
neighboring  decisions.  Several  thousand  segments  are 
used  for  each  training  class,  and  an  equal  number  for  each 
test  class. 

For  each  detector,  the  final  feature  subset  was  deter¬ 
mined  by  an  exhaustive  search  for  subsets  of  plausible 
features.  Each  possible  subset  was  trained  and  then  tested 
on  independent  sets  of  hand-marked  data  from  one  or  more 
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subjects.  The  resulting  networks  were  then  used  for  clas¬ 
sification  with  other  data  from  the  same  or  new  subjects. 
The  system  frequently  finds  contaminants  which  would  be 
difficult  to  detect  with  any  static  setting  of  signal  ampli¬ 
tude  thresholds.  Fig.  3  shows  a  sample  eye-blink  contam¬ 
inant  and  two  channels  of  time  series  which  were  marked 
by  the  system.  While  the  first  channel  could  easily  be  re¬ 
jected  by  a  simple  fixeu-amplitude  threshold  detector,  the 
second  contains  contaminated  variance  “buried”  in  the 
neuroelectric  signals. 

In  practice,  the  automated  system  performs  a  first  pass 
through  the  data,  replacing  one  of  the  two  human  scorers 
who  formerly  hand  marked  the  data.  A  human  scorer  then 
rapidly  reviews  the  system’s  decisions.  Routine  use  of  this 
system  over  the  last  two  years  has  reduced  by  a  factor  of 
15  the  man  hours  required  to  edit  contaminated  data  from 
recordings  with  up  to  64  channels. 

2)  Evoked  Potential  Estimation:  Evoked  potentials 
(also  called  event-related  potentials  or  ERP’s)  are  neuroe¬ 
lectric  signals  which  are  time  registered  to  an  experimen¬ 
tal  stimulus  or  a  subject’s  response.  Because  the  ampli¬ 
tude  of  these  stimulus  or  response-related  neuroeiectric 
signals  can  be  smaller  than  the  neuroeiectric  activity  un¬ 
related  to  the  experimental  events,  signal  processing  must 
be  used  to  extract  estimates  of  their  waveforms.  For  those 
cases  where  the  timing  and  morphology  of  each  evoked 
potential  has  some  similarity  over  many  stimulus  presen¬ 
tations  (trials),  a  simple  average  will  improve  the  signal- 
to-noise  ratio  by  a  factor  of  the  square  root  of  the  number 
of  trials.  However,  real  evoked  potentials  frequently  vary 
significantly  in  both  respects,  leading  to  a  requirement  of 
many  stimulus  repetitions  before  a  useful  average  can  be 
obtained.  Many  methods  exist  for  computing  improved 
estimates  of  averaged  event-related  potentials,  given  some 
assumptions  about  the  underlying  signal  characteristics 
[17].  For  many  of  our  experiments  on  higher  cognitive 
brain  functions,  we  cannot  assume  that  stimulus-related 
signals  are  measurable  in  every  trial  nor  can  we  presume 
much  knowledge  about  statistical  properties  of  event-re¬ 
lated  signal  and  unrelated  “noise.”  We  have  developed  a 
method  using  NN’s  for  averaged  ERP  estimation  without 
the  first  assumption,  and  with  minimal  assumptions  about 
signal  and  noise  properties  [9].  In  particular,  we  assume 
that  the  noise  process  is  roughly  stationary,  and  the  non¬ 
stationary  signal  process  can  be  time  registered  to  an  ex¬ 
ternal  event.  We  can  then  synthesize  a  stationary  noise 
process  with  statistics  very  close  to  those  of  the  actual 
noise  by  randomizing  the  start  point  of  the  analysis  inter¬ 
val  for  each  trial.  Since  the  nonevent-related  potentials 
are  stationary  and  the  power  in  an  individual  trial  is  pri¬ 
marily  nonevent- related  (SNR  «  1  over  the  trial),  the 
synthesized  ensemble  has  statistics  quite  close  to  those  of 
the  “noise.  "  Discriminability  of  the  time- registered  trials 
from  this  ensemble  can  be  taken  as  a  measure  of  signal 
strength. 

We  have  used  NN  procedures  to  select  trials  with  dis- 
criminable  event-related  signals  in  a  defined  time  interval 
by  attempting  to  discriminate  the  set  of  time-registered 


FT 
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Fig.  3.  Three  channels  of  neuroeiectric  time  senes  with  eye  blink  contam¬ 
inant  delected  by  NN  system  of  Fig.  I.  (t)  The  vertical  eye  movement 
nme  senes  (VEO)  clearly  shows  a  blink  O.fi-O.8  s  past  the  stimulus  on¬ 
set.  (b)  Left  frontal  neuroeiectric  channel  (FI)  shows  a  large  eye-con¬ 
taminant  wave  which  could  easily  be  detected  with  a  simple  fixed-thresh¬ 
old  algonlhm.  (c)  Antenor  midline  precentral  neuroeiectric  channel 
(oCc)  is  also  contaminated  by  the  eye  blink,  but  the  maximal  or  inte¬ 
grated  amplitudes  would  not  trigger  an  amplitude  threshold  device  since 
the  just-pnor  uncontaminated  signal  has  a  larger  amplitude  than  the  con¬ 
taminant. 
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m  annum 


Fig  4  NN  eveni-rclated  potential  (HRP)  estimation. 


trials  from  the  set  of  artificially  made  “noise”  trials  (Fig. 
4).  The  layered-network  classifier  of  Fig.  1  is  used  to  find 
those  trials  (time  registered  to  a  stimulus)  which  have  a 
consistent  event- related  signal  in  an  interval  of  interest. 
This  is  done  by  comparing  the  ensemble  of  candidate  trials 
to  the  ensemble  of  "noise”  trials  synthesized  from  the 
same  data.  “Enhanced”  averages  are  then  formed  from 
the  selected  signal-bearing  trials. 

For  each  neuroelectnc  signal  channel,  an  equation  was 
derived  which  discriminates  the  “noise”  trials  by  apply¬ 
ing  the  NN  algorithm  to  a  prelabeled  training  set  consist¬ 
ing  of  two-thirds  of  the  total  trials.  The  equation  was 
tested  on  the  remaining  third  of  the  trials.  This  procedure 
was  repeated  for  each  third  of  the  data  (using  the  remain¬ 
ing  two-thirds  for  training).  This  multiple  partitioning,  or 
jackknife  cross  validation,  reduced  sampling  error  due  to 
test  set  selection.  If  overall  classification  was  significant, 
trials  which  were  classified  as  containing  event-related 
signals  were  then  selected.  The  averages  computed  from 
these  trials  show  enhancement  of  evoked  potential  peaks 
in  the  chosen  interval  (Fig.  5).  In  the  example  shown  in 
the  figure,  the  positive  peak  391  ms  after  the  stimulus 
(P391)  is  31  percent  larger  in  the  enhanced  average.  The 
averages  of  unselected  trials  are  relatively  flat  in  that  in¬ 
terval. 

Thus,  using  an  NN  procedure  to  reject  realizations  with- 
a  low  signal-to-noise  ratio,  the  estimate  of  a  consistent 
signal  component  may  be  enhanced  without  prior  assump¬ 
tions  about  the  statistical  properties  of  the  signal  and  noise 
inherent  in  techniques  such  as  minimum  mean-square  er¬ 
ror  (MMSE)  filtering.  When  knowledge  of  the  signal  is 
limited,  this  approach  can  still  be  used  to  pick  realizations 
which  maximally  differ  from  the  ongoing  noise. 

B  Feature  Extraction 

Neuroelectric  signals  are  sensitive  indexes  of  the  cen¬ 
tral  nervous  system  effects  of  psychotropic  medications, 
and  have  proven  useful  in  distinguishing  broad  classes  of 


Fig.  5.  Averaged  stimulus-registered  visual  ERP  waveforms  obtained  by 
presenting  a  stimulus  85  times  (top),  enhanced  average  resulting  from 
NN  selection  of  those  stimulus  trials  with  delectable  event-related  brain 
signals  (center),  and  average  of  tnals  without  detectable  signals  (bot¬ 
tom). 


drugs  such  as  stimulants,  sedatives,  antipsychotics,  etc., 
[14],  [15].  The  problem  of  distinguishing  subtypes  of  the 
same  class  of  drugs  is  more  difficult,  however.  As  in  other 
applications,  the  major  problem  is  choosing  the  most  ef¬ 
fective  subset  of  signal  features  from  a  large  candidate 
population.  As  a  contribution  to  this  area,  we  have  used 
NN  techniques  to  select  subsets  of  neuroelectric  spectra! 
features  which  could  distinguish  each  of  three  benzo¬ 
diazepine  sedative  hypnotic  drugs  (sleeping  pills)  from  a 
placebo  [5].  This  resulted  in  a  set  of  feature  “signatures” 
which  characterize  the  effects  of  each  of  these  drugs. 

The  effects  of  multiple  therapeutic  doses  of  three  ben¬ 
zodiazepines  (quazepam,  triazolam,  and  flurazepam)  on 
periods  of  slow  wave,  nonrapid-eye-movement  sleep  were 
assessed.  Such  periods  from  all-night  sleep  recordings  of 
eight  healthy  men  were  spectral  analyzed,  and  the  data  of 


CEVINS  AND  MORGAN  NN  SIGNAL  PROCESSING  IN  BRAIN  RESEARCH 


1157 


each  drug  were  compared  to  the  placebo  using  the  NN 
algorithm  described  above.  Equations  were  developed  and 
validated  which  characterized  the  effects  of  the  drugs  on 
the  eight  subjects. 

Spectral  bins  from  0.5  to  15.5  Hz  in  0.5  Hz  steps  rep¬ 
resented  the  average  spectral  intensity  for  a  specific  fre¬ 
quency  component  during  a  given  16  s  interval.  Fig.  6 
shows  stacked  spectral  displays  for  the  three  drugs  and 
the  placebo  from  one  of  the  eight  subjects.  A  decrease  of 
the  delta  and  theta  bands  (0.5-3. 5  and  4-7.5  Hz.  respec¬ 
tively  )  and  increase  of  the  sigma  band  (12-14.5  Hz) 
spectral  intensity  existed  for  all  drugs.  However,  it  is  dif¬ 
ficult  to  make  precise  comparisons  between  the  drugs  from 
this  sort  of  visual  display,  and  the  eight  subjects  differed 
considerably.  Furthermore,  the  mean  spectral  intensities 
were  not  significantly  different  between  drugs. 

Using  the  layered-network  algorithm  described  above, 
we  determined  which  of  the  seven  0.5  Hz  components  of 
each  of  three  conventional  frequency  bands  (called  delta, 
theta,  and  sigma)  were  optimal  for  discriminating  each 
drug  from  placebo.  Then  the  seven  best  frequency  com¬ 
ponents  (three  delta,  two  theta,  two  sigma)  were  com¬ 
bined  in  a  single  analysis  called  the  ‘‘full-spectrum" 
analysis. 

For  each  of  the  four  sets  of  spectral  measures  (delta, 
theta,  sigma,  full  spectrum),  for  each  of  the  nine  hy¬ 
potheses  (each  of  the  three  drugs  versus  placebo  in  each 
third  of  the  evening),  each  subject’s  drug  and  placebo 
spectral  intensity  observations  were  converted  into  stan¬ 
dard  scores  ( 0  mean  and  unit  variance )  in  order  to  reduce 
the  effect  of  irrelevant  differences  in  overall  signal  ampli¬ 
tude  between  subjects.  Each  standardized  spectral  esti¬ 
mator  for  each  drug  was  then  discriminated  from  the  pla¬ 
cebo,  yielding  a  percent  accuracy  of  discrimination.  The 
discriminating  equations  were  validated  using  three  leave- 
out-one-third  replications  as  described  in  the  section  on 
evoked  potential  estimation.  The  significance  of  the  clas¬ 
sification  accuracies  was  determined  by  reference  to  the 
mean  and  variance  of  nine  '‘pseudoclassifications. "  A 
pseudoclassification  consisted  of  attempting  to  discrimi¬ 
nate  the  placebo  spectral  observations  of  one-third  of  the 
night  from  those  of  another  third.  An  index  of  a  drug’s 
effect,  from  0  to  100  percent,  was  formed  by  subtracting 
the  expected  placebo-versus-placebo  from  the  drug-ver- 
sus-placebo  classification  accuracy,  and  then  dividing  the 
maximum  possible  drug-versus-placebo  minus  placebo- 
versus-placebo  accuracy.  For  example,  if  the  drug-ver¬ 
sus-placebo  accuracy  was  75  percent,  and  if  the  placebo- 
versus-placebo  accuracy  was  51  percent,  then  the  percent 
drug  effect  would  be  (75-51 ) /( 100-51  )  =  49  percent. 

The  spectral  signature  of  each  drug  was  determined  by 
extracting  the  weightings  given  to  the  individual  fre¬ 
quency  components  of  significant  drug-versus-placebo 
equations.  This  was  done  by  weighting  each  feature  in  an 
equation  by:  1)  the  relative  size  of  its  discriminant  func¬ 
tion  term.  2)  the  relative  size  of  its  TLU,  and  3)  the  neg¬ 
ative  log  of  the  significance  of  the  equation. 

All  three  drugs  differed  markedly  from  the  placebo  dur- 
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Fig.  6.  Slacked  spectral  display  of  slow-wave  sleep  of  Subject  5  after 
administration  of  a  placebo  la),  and  three  benzodiazepine  sleeping  pills 
(bHd).  Each  spectral  line  represents  16  s  of  slow-wave  sleep 
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ing  each  third  of  the  evening.  Flurazepam  had  the  largest 
effect  on  the  full  spectrum  of  neuroelectric  frequency 
components;  quazepam  and  triazolam  had  smaller,  equal 
effects.  All  drugs  reduced  delta  and  theta  band,  and  in¬ 
creased  sigma  band  spectral  intensity,  but  to  differing  de¬ 
grees  and  with  their  own  characteristic  spectral  signa¬ 
tures.  For  instance,  quazepam's  signature  (Fig.  7)  was  a 
predominant  increase  in  the  12  Hz  and  a  decrease  in  the 
7.5  Hz  components.  The  derivation  of  such  a  “spectral 
signature"  as  well  as  a  single  index  of  the  overall  drug 
effect  is  characteristic  of  NN  signal  processing  analysis 
methods;  the  features  chosen  for  discrimination  can  them¬ 
selves  be  as  instructive  as  the  resulting  classification.  In 
particular,  since  the  candidate  features  were  transformed 
to  standard  scores,  the  feature  weights  suggest  the  relative 
importance  of  each  feature. 

C.  Pattern  Classification  and  Brain  State  Prediction 

The  measurement  of  spatiotemporal  neuroelectric  pat¬ 
terns  related  to  higher  cognitive  brain  functions  is  a  par¬ 
ticularly  difficult  signal  processing  problem.  The  essential 
difficulty  is  a  low  SNR  confounded  by  a  plethora  of  pos¬ 
sible  signal  features.  In  applying  NN  techniques,  the  goal 
is  to  physically  interpret  the  selected  features,  the  classi¬ 
fication  equations,  and  the  relative  classification  perfor¬ 
mance  between  spatial  and  temporal  constraint  groups. 
When  NN's  are  used  to  study  the  brain  mechanisms  of 
higher  cognitive  functions,  interpretation  of  the  classifi¬ 
cation  equations  can  be  more  important  than  optimizing 
the  accuracy  of  classification  for  differing  cognitive  tasks. 

In  our  current  use  of  NN  to  study  cognitive  functions, 
inputs  to  the  networks  consist  of  parameters  derived  from 
brain  signals  measured  at  up  to  64  locations  on  the  scalp 
of  people  playing  specially  contrived  video-type  games. 
In  a  typical  experiment,  about  a  gigabyte  of  raw  brain  data 
is  recorded,  from  which  several  thousand  replications  of 
a  thousand  or  so  signal  features  are  computed.  Outputs  of 
the  networks  consist  of  decisions  as  to  which  functional 
brain  state  the  brain  signal  parameters  represent.  The  con¬ 
nections  and  weights  of  the  networks  are  interpreted  to 
answer  hypotheses  about  brain  function.  Neurological 
interpretation  of  the  network  parameters  is  made  possible 
by  using  specific  constraints,  based  on  the  anatomy  of  the 
cerebral  cortex,  for  each  of  the  networks.  These  con¬ 
straints  also  permit  an  exhaustive  search  for  globally  op¬ 
timal  TLU's  by  reducing  the  number  of  combinations  of 
brain  signal  features  to  a  computable  amount. 

Fig.  8  shows  the  results  of  this  analysis  in  a  recent  ex¬ 
periment  [10],  [12],  By  extracting  just  those  brain  signals 
related  to  subjects'  preparation  to  react  to  a  number  on  a 
video  screen,  we  found  that  certain  areas  of  the  brain  ap¬ 
peared  to  be  coordinating  themselves  in  anticipation  of 
the  forthcoming  numeric  stimulus.  We  call  these  patterns 
“preparatory  neural  networks.”  and  they  are  shown  in 
Fig.  8  as  hatched  lines  connecting  different  points  on  the 
head.  The  areas  of  the  brain  that  appear  to  be  important 
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Fig.  7.  (»>-(c)  Characteristic  spectral  signatures  of  drug  effects  obtained 
for  each  drug  from  the  coefficients  of  the  classification  equations  Ordi¬ 
nate  is  dimensionless  relative  weighting  of  spectral  features 

in  preparing  for  good  performance  are  known  as  the  left 
prefrontal  cortex  (the  “cognitive  sequencing"  area),  the 
midline  premotor  cortex  rihe  area  which  is  the  “execu¬ 
tive"  of  all  voluntary  movements),  and  the  motor  and 
parietal  cortices  (areas  involved  in  controlling  finger 
movements  and  feeling  the  pressure  of  the  response  but- 
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Fig.  8.  Patterns  of  "preparatory  neural  networks" showing  apparent  co¬ 
ordination  of  certain  areas  of  the  brain  in  anticipation  of  a  forthcoming 
numeric  stimulus 

ton  against  the  finger).  When  these  areas  are  not  involved 
or  not  well  coordinated  during  preparation,  the  subject’s 
performance  is  likely  to  be  inaccurate.  Many  errors,  it 
seems,  are  predetermined  by  the  failure  to  establish  a 
proper  preparatory  network  during  the  split  second  before 
the  task  These  preparatory  .networks  are  thus  an  indica¬ 
tion  of  the  importance  of  the  human  brain’s  ability  to  cre¬ 
ate  and  constantly  update  models  of  itself  and  the  external 
world. 

Obtaining  this  result  required  using  the  NN  procedure 
in  a  number  of  ways.  First,  trials  with  detectable  signals 
were  selected  using  the  evoked  potential  enhancement 
technique  described  earlier.  After  a  spatial  Lapiacian  was 
calculated  for  each  channel,  between-channel  covariances 
were  computed  across  375  ms  intervals  for  trials  low  pass 
filtered  at  3  Hz.  Covariances  for  decimated  time  points 
prior  to  the  stimulus  in  each  trial  were  considered  possible 
features  to  distinguish  subsequent  performance  accuracy. 
The  combinatorial  problem  of  selection  of  a  subset  of 
these  features  was  reduced  to  only  considering  those  co- 
variances  which  were  statistically  significant  in  a  previous 
analysis  of  filtered,  enhanced  averages.  Analyses  were 
done  for  seven  subjects,  both  as  a  group  (equations  formed 
from  six  of  the  subjects,  and  tested  on  the  seventh),  and 
as  individuals  (classification  of  one-fifth  of  the  trials  using 
equations  from  the  other  four-fifths,  repeated  over  ail 
fifths).  The  overall  discrimination  of  subsequently  accu¬ 
rate  from  subsequently  inaccurate  trials  for  the  group  was 
59  percent,  which  was  significant  at  the  0.01  level  for  the 
number  of  feature  patterns  used.  For  the  one  subject  with 
most  trials,  the  average  correct  classification  was  68  per¬ 
cent,  which  was  significant  at  the  0.001  level.  In  other 
words,  given  a  general  superset  of  likely  features,  the  NN 
algorithm  was  able  to  choose  features  which  in  combi¬ 
nation  would  predict  subsequent  accuracy  of  performance 
on  the  task  up  to  about  two-thirds  of  the  time.  Both  the 
significant  levels  of  these  classifications  and  the  spatial 
specificity  of  the  features  selected  to  achieve  the  classifi¬ 
cation  were  consistent  with  a  view  that  specific,  spatially 
coordinated  “neural  preparatory  sets"  are  an  important 
determinant  of  the  accuracy  of  human  visuomotor  perfor¬ 
mance. 

In  another  experiment,  we  found  neuroelectnc  pattern 
changes  which  preceded  deterioration  of  performance  as 


subjects  performed  difficult  concentration  tasks  for  many 
hours  [13J.  Five  fighter  test  pilots  performed  difficult  vi- 
suomotor-memory  tasks  during  a  10-14  h  period.  The  NN 
pattern  classification  methods  described  above  were  used 
to  measure  the  effects  of  prolonged  concentration,  and 
particularly  to  search  for  neuroelectric  indicators  which 
predicted  the  onset  of  deteriorating  performance.  The  time 
series  for  each  single  trial  was  processed  with  a  spatial 
Lapiacian,  and  with  a  temporal  low-pass  filter  with  a  cut¬ 
off  of  3  Hz.  Decimated  intervals  of  length  0.5  s  from  each 
channel’s  time  series  were  used  to  compute  covariances 
between  all  pairwise  combinations  of  the  18  neuroelectric 
signal  channels.  In  the  pilot  with  the  most  data,  leading 
indicators  were  found  which  distinguished  fully  aleit  and 
incipiently  impaired  sets  of  trials  with  equally  accurate 
performance  81  percent  of  the  time  (average  of  five  leave- 
out-one-fifth  validations).  This  initial  result  suggested  that 
similar  methods  could  be  used  to  develop  an  on-line  warn¬ 
ing  system  so  that  people  engaged  in  hazardous  or  critical 
work  would  be  alerted  to  the  possibility  of  impaired  per¬ 
formance  due  to  mental  fatigue. 

Discussion 

We  have  described  several  applications  of  NN  signal 
processing  methods  in  which  success  of  the  application 
required  more  than  a  clever  NN  algorithm.  In  particular, 
expert  specification  of  application-specific  constraints, 
statistical  procedures  for  generating  reference  noise  dis¬ 
tributions,  and  statistical  procedures  for  checking  validity 
and  reliability  were  required.  NN  algorithms,  when  cou¬ 
pled  with  these  and  other  analogous  methods,  may  prove 
to  be  generally  useful  in  signal  processing.  They  can  aiso 
serve  as  an  explicit  description  of  the  parallel  implemen¬ 
tation  of  pattern  recognition  algorithms  for  a  hardware  re¬ 
alization. 

In  considering  these  methods,  we  should  not  confuse 
the  term  “neural  network”  with  how  a  human  brain  ac¬ 
tually  works.  The  human  brain  accomplishes  its  prodi¬ 
gious  feats  of  perceiving,  thinking,  and  acting  with  some¬ 
where  between  10"-1012  processing  elements  (neurons). 
A  substantial  subset  of  these  elements  (the  pyramidal  cells 
of  the  neocortex  with  their  complex  processing  capabili¬ 
ties  and  connections  with  up  to  50  000  other  neurons)  are 
more  comparable  to  mainframe  computers  than  they  are 
to  the  op-amp  model  neurons  of  current  NN  algorithms. 
The  human  brain’s  processing  elements  are  organized  into 
several  hundred  major  divisions,  each  comprised  of  mil¬ 
lions  to  billions  of  neurons.  These  divisions  are  in  turn 
organized  into  a  few  dozen  major  functional  systems  that 
perform  such  functions  as  analysis  of  light  and  sound  and 
pressure  to  the  body  surface,  control  of  muscles,  linguis¬ 
tic  encoding  and  decoding,  and  generation  of  models  of 
self  and  world.  Adaptive,  purposive  behavior  is  found  at 
ail  levels  of  organization  of  the  brain,  down  io  the  indi¬ 
vidual  neurons  themselves.  Roughly  three-fourths  of  the 
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brain  is  concerned  with  higher  associative  and  executive 
functions  not  direcdy  connected  with  processing  sensory 
data  and  executing  motor  responses. 

Consideration  of  the  human  brain  thus  suggests  the  need 
for  quotation  marks  when  referring  to  NN  signal  process¬ 
ing  and  pattern  recognition  systems  as  being  “brain-like.” 
Although  there  is  no  harm  in  giving  anthropomorphic 
names  to  our  algorithms  and  systems,  it  is  not  helpful  in 
the  long  run  to  lose  sight  of  the  vast  difference  in  scale 
and  capability  between  the  machines  we  create  and  the 
apparatus  with  which  we  create  them.  Our  AI  and  our  NN 
systems  have  “intelligence"  in  the  same  sense  that  a 
broad  jumper  could  said  to  be  flying  (Chomsky,  personal 
communication,  1984). 

The  lesson  from  real  brains  for  designing  better  NN  sig¬ 
nal  processing  systems  is  the  apparent  necessity  of  devot¬ 
ing  a  major  amount  of  resources  in  a  self-organizing  sys¬ 
tem  to  higher  level  executive  functions  that  generate 
models  of  the  system  and  the  world  external  to  it.  In  prac¬ 
tical  terms,  this  suggests  a  union  of  AI  and  NN  systems, 
the  context  and  goal  direction  being  specified  by  the  for¬ 
mer  and  feature  extraction  and  pattern  recognition  by  the 
latter.  In  a  limited  way,  such  a  union  was  required  to  make 
each  of  the  sample  applications  reported  here  work  well 
enough  to  be  practical. 

Conclusions 

We  have  reported  four  applications  in  brain  research  of 
NN  signal  processing  and  feature  analysis: 

1)  contaminant  detection  for  replacement  of  ad  hoc  de¬ 
tectors, 

2)  waveform  detection  for  evoked-potential  estima¬ 
tion. 

3)  feature  extraction  for  identification  of  characteristic 
“signatures”  of  different  sleeping  pills, 

4)  pattern  classification  and  state  prediction  for  mea¬ 
suring  human  cognitive  brain  functions. 

Analyses  in  these  areas  have  traditionally  depended 
heavily  upon  either  expen  selection  of  features  or  on  rote 
application  of  statistical  methods.  The  major  significance 
of  NN  methods  is  that  they  provide  an  effective  way  of 
incorporating  expen  knowledge  in  a  mathematical  frame¬ 
work.  The  resulting  blend  has  solved  problems  that  were 
difficult  to  solve  previously.  The  approach  described  here 
may  well  be  applicable  to  many  problems  in  signal  inter¬ 
pretation. 

Finally,  we  summarized  the  implications  of  current 
knowledge  about  brain  function  for  computer  engineers 
interested  in  using  artificial  neural  network  signal  pro¬ 
cessing. 

1)  10  000  op-amps  do  not  make  a  brain.  A  better  elec¬ 
tronic  analogy  of  a  human  brain  would  be  a  hundred  bil¬ 
lion  mainframes,  essentially  globally  connected,  each 
running  the  correct  program. 

2)  While  low-level  networks  are  important  for  both 
neural  systems  and  engineering  applications,  the  example 
of  the  human  brain  suggests  that  self  and  world  models 


generated  by  high-level  executive  systems  are  critical  to 
system  function. 
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In  seven  right-handed  adults,  the  brain  electrical  patterns  before  accurate  performance 
differed  from  the  patterns  before  inaccurate  performance.  Activity  overlying  the  left 
frontal  cortex  and  the  motor  and  parietal  cortices  contralateral  to  the  performing  hand 
preceded  accurate  left-  or  right-hand  performance.  Additional  strong  activity  ovcrlving 
midline  me  tor  and  premotor  cortices  preceded  left-hand  performance.  These  measure¬ 
ments  suggest  that  brief,  spatially  distributed  neural  activity  patterns,  or  “preparatory 
sets,”  in  distinct  cognitive,  somesthetic-motor,  and  integrative  motor  areas  of  the 
human  brain  may  be  essential  precursors  of  accurate  visuomotor  performance. 


Preparatory  set  for  human  vi¬ 
suomotor  performance,  defined  as  a 
state  of  readiness  to  receive  a  stimulus 
or  make  a  response  (I),  has  been  studied  by 
a  vanetv  of  disaplines.  Temporal  properties 
of  preparatory  sets  have  been  measured  in 
information- processing  studies,  but  such 
studies  have  not  focused  on  the  underlying 
neural  systems  (2\.  Spanal  properties  of 
preparatory  sets  measured  in  cerebral  blood 
flow  studies  have  revealed  increased  meta¬ 
bolic  activity  for  sensory- specific  focus  of 
attention  in  superior  prefrontal,  midfrontal, 
and  anterior  parietal  cortices  (3).  These 
studies  have  been  umited.  however,  bv  the 
temporal  resolution  il  minute  or  longer)  ol 
blood  flow'  measurement  techniques.  Clini¬ 
cal  neuropsychological  studies  have  demon¬ 
strated  that  behaviors  requiring  preparatory 
sets  \4 1  reiv  on  intact  lateral  frontal  regions 
(5),  but  variability  m  size  and  locaoon  of 
lesions  has  limited  the  spanal  specificity  of 
such  studies  in  localizing  normal  funenon. 
And  although  scalp- recorded  brain  electrical 
and  magneuc  recordings  provide  both  spa¬ 
tial  and  temporal  information  on  neural 
acnvity  underhing  preparatory  sets,  studies 
of  the  contingent  negative  varianon  (CNV), 
an  event- related  brain  potential  component 
thought  to  be  related  to  preparatory  set, 
have  often  yielded  controversial  or  ambigu¬ 
ous  results  (6). 

Recording  from  26  electrodes  and  using 
several  signal -enhancing  procedures,  we 


measured  tne  rapidly  changing  spanal  pat¬ 
terns  of  mass  neuroclccmc  activity  associat¬ 
ed  with  preparanon  and  cxecuuon  of  precise 
nght-  and  left-hand  finger  pressures  in  re¬ 
sponse  to  visual  numeric  stimuli.  We  found 
differences  occurring  during  the  presumulus 
period  between  patterns  associated  with 
subsequently  accurate  and  inaccurate  per¬ 
formance.  These  group  differences  allowed 
disenminanon  of  subsequent  performance 
accuracy  for  both  hands  of  individual  sub¬ 
lets.  Thus,  a  spanallv  specific,  multicom¬ 
ponent  neural  preparatory  set,  composed  of 
an  invariant  left  frontal  component  and 
hand- specific  central  and  parietal  compo¬ 
nents,  mav  be  essential  for  accurate  perform¬ 
ance  of  certain  rvpes  of  difficult  visuomotor 
tasks. 

Seven  healthy,  right-handed  male  adults 
were  recruited  from  the  communin'  and 
paid  for  their  participation.  Thev  were  re¬ 
quired  to  exert  rapid,  preaseiv  graded  pres¬ 
sures  ( forces  from  0.1  to  0.9  kg)  followed  bv 
immediate  release,  with  nght-  and  left-hand 
index  fingers  in  response  to  visual  numcnc 
stimuli  (numbers  1  to  9).  The  stimulus  was 
presented  randomly  on  successive  trials  1 
second  after  a  cue  (the  letter  V  lasting  0.3 
second)  that  was  slanted  at  a  30°  angle  to  the 
nght  or  left  to  indicate  the  required  re¬ 
sponse  hand  ;~i.  In  “respond”  trials,  the 


EEG  Svstcms  Laboratory.  1855  Foisom  Street.  San 
Francisco.  CA  1 03 


SCIENCE.  VOL 


FINGER  RESPONSE 


uV  /  cm2 
0.029 


*  >  s 


-0.078  # 

.05  10-3  10 

Significance 


COVARIANCE 


3FIIUr 

R»3pon3»: 

-32  to  155 

mu  c 

EECSL 


Flg.1  .  Most  significant,  bctween-channei  covariance  patterns  (colored  lines i,  looking  down  at  the  top 
of  the  head,  from  the  wave  at  the  peak  of  the  response  superimposed  on  colored  maps  of  that  wave's 
amplitude.  The  motor-related  wave  was  measured  during  a  187-msec  interval  centered  on  the  peak  of 
left-hand  and  right-hand  index  anger  pressures  from  seven  neht-handed  men  ( 19).  The  thickness  of  a 
covariance  line  is  proportional  to  the  negative  logarithm  of  its  significance  (from  0.05  to  0.00005)  {17). 
A  violet  line  indicates  a  positive  covariance  ( motor-related  waves  with  the  same  polantv),  and  a  blue  line 
indicates  a  negative  covariance  (motor- related  waves  with  opposite  polannes).  The  color  scale  at  the 
left,  representing  wave  amplitude,  covers  the  range  from  the  minimum  to  maximum  values  of  the  two 
maps.  .Ml  covariances  refer  to  the  site  overlving  supplementary  and  premotor  comces.  There  is  a  strong 
lateralization  of  frontal,  central,  and  antcro- parietal  covariances  over  the  hemisphere  contralateral  to  the 
responding  hand,  a  result  consistent  with  the  lateralization  of  the  amplitude  maps. 


stimulus  was  slanted  in  the  same  direction  as 
the  cue,  and  the  subject  was  to  respond 
quickly  with  linger  pressure  of  the  indicated 
hand,  with  a  force  corresponding  to  the 
stimulus  number  on  a  linear  scale  from  1  to 
9.  In  a  random  20%  of  the  trials,  the 
stimulus  was  slanted  opposite  to  the  cue  and 
the  subject  was  to  make  no  response.  These 
miscued  “catch”  trials  ensured  that  subjects 
attended  to  the  cues  and  stimuli.  To  help 
subjects  calibrate  then  icsponscs,  the  pres¬ 
sure  produced  was  displayed  1  second  after 
completion  of  each  response  (<?). 

Brain  potentials  from  26  scalp  electrodes 
(9),  vertical  and  horizontal  eve  movement 
potentials,  and  flexor  digitori  muscle  poten¬ 
tials  were  recorded  onto  magnetic  tape  at 
128  Hz  from  0.73  second  before  chc  cue  to 
1  second  after  feedback  ,  10).  The  Laplacian 
operator,  a  spatial  pattern  enhancement 
technique,  was  applied  to  che  brain  poten¬ 
tials  at  every  time  point  to  reduce  the  blur 
distortion  that  results  as  potentials  are  trans¬ 
mitted  from  the  brain  to  che  scalp  ( 11 ).  Two 
independent  raters  edited  the  data  for  arti¬ 
facts  bv  visual  inspection  of  brain,  eve  move¬ 
ment,  and  muscle  potential  polygraph  chan¬ 
nels.  Tnais  with  artifacts  due  to  eve  move¬ 
ment,  head  or  electrode  movement,  or  scalp 
muscle  contamination  were  eliminated,  as 
were  trials  with  slow,  bimodai.  or  delayed 
responses,  or  with  flexor  digitorum  activity 
between  chc  cue  and  chc  stimulus. 

The  remaining  tnais  (60%)  were  then 
sorted  for  response  accuracy,  and  the  two 
sets  of  tnais  were  balanced  according  to  a 
number  of  catena  to  avoid  confounding 
performance  vananons  due  to  transitory  and 
longer  lasting  changes  in  arousal  and  learn¬ 
ing  with  inaccuracv  per  se.  Accurate  and 
inaccurate  data  sets  consisted  of  tnais  in 
which  the  error  (deviation  from  required 
pressure;  for  each  subject  was,  respectively, 
less  and  greater  than  his  mean  error  over  all 
remaining  tnais  1 12).  Mean  reaction  time, 
averaged  across  all  subjects,  was  consistent 
among  hand  and  accuracy  conditions  (610 
to  618  msec). 

To  quantitate  the  electrical  activity  of  the 
brain,  we  measured  the  covanance  (similar¬ 
ity  of  wave  shape)  between  different  pairs  of 
electrodes  over  bnef  segments  (187  or  375 
msec)  of  event-related  (cue,  stimulus,  re¬ 
sponse,  feedback)  waveforms  averaged  from 
the  seven  subjects  ( 13-15).  Covanances  be¬ 
tween  each  of  the  120  combinations  of  the 
16  Laplacian-transformed  channels  were 
computed  from  enhanced  (16)  and  filtered 
average  waveforms.  We  determined  the  co¬ 
vanance  for  each  electrode  pair  by  comput¬ 
ing  the  cross-covanance  function  between 
their  waveform  segments,  with  the  lag  time 
for  one  channel  with  respect  to  the  other 
varvwg  from  0  to  125  msec.  The  value  of 


covariance  was  the  maximum  absolute  value 
of  that  function.  For  the  wave  at  the  peak  of 
the  response,  covanances  were  analyzed  to 
determine  whether  thev  were  significantly 
different  from  noise  values  (17.  IS);  we 
could  then  compare  the  levels  of  significance 
of  each  electrode  pair  under  different  expen- 
mental  conditions. 

To  validate  the  analysis  in  a  known  case, 
this  procedure  was  applied  to  waveforms 
time-registered  to  chc  onset  of  che  finger 
pressure  response.  The  most  significant  Icft- 
and  right-hand  covanances  occurred  be¬ 
tween  electrodes  overlying  cortical  regions 
involved  in  motor  execution  (Fig.  1)  {19). 
These  patterns  of  covanance  presented 
much  more  spatially  discrete  information 
chan  their  corresponding  amplitude  maps 
(20).  In  chc  187-msec  interval  centered  on 
the  peak  of  the  response  (62  msec  after 
response  onset),  right-  and  left-hand  covan¬ 
ance  patterns  were  nearly  mirror  images.  In 
both  patterns  ail  covanances  involved  the 
midline  antero-central  site  overlving  the  pre¬ 
motor  and  supplementary  motor  comces. 
Covanances  between  this  site  and  the  left 
frontal,  antero-central,  central,  and  antero- 
panetal  sites  for  nght-hand  responses,  and 
between  corresponding  nght-hemispherc 


sites  (  except  nght  antero-central)  for  the  left 
hand,  were  all  consistent  with  known  mo¬ 
tor-related  cortical  areas. 

The  procedure  was  then  applied  to  the 
cue- co- stimulus  penod  to  study  preparatory 
sets  i21).  Stansoca!  comparison  of  the  CNV 
amplitudes  (Fig.  2)  during  an  interval  500 
to  875  msec  after  the  cue  did  not  reveal 
significant  differences  between  accurate  and 
inaccurate  conditions  (22).  During  this 
same  penod,  however,  well-defined  be- 
tween-channci  covanance  patterns  relaccd  to 
subsequent  accuracy  were  discovered.  Thev 
first  appeared  in  the  interval  centered  500 
msec  after  the  cue  and  became  well  differen¬ 
tiated  between  accurate  and  inaccurate  con¬ 
ditions  in  the  500-  to  875-msec  interval 
(centered  313  msec  before  stimulus  onset) 
spanning  the  late  component  of  the  CNV, 
The  lack  of  muscle  potential  and  eve  move¬ 
ment  signals  in  these  intervals  confirmed 
that  these  patterns  were  neural  in  ongin 
(Fig.  3). 

Covanance  patterns  during  the  penod 
between  the  cue  and  the  stimulus  (Fig.  4) 
were  distinct  from  those  related  to  overt 
finger  responses.  During  the  interval  from 
500  to  875  msec  after  uie  cue  onset,  covan¬ 
ance  patterns  associated  with  subsequently 


30  JANUARY  198- 


REPORTS  sSi 


Accurate  trials  Inaccurate  trials 


0  Q.25  0.5  0.75  1  1.25  0  0.25  0.5  0.7S  1  1.25 

Time  (seconds) 


Fig.  2.  Amplitudes  ot  the  CNV  computed  during  the  cue-to-stimuJus  period.  .Amplitudes  between  low- 
pass  filtered  (below  3  Hzi.  event- related  Laplacian  waveforms,  averaged  from  seven  sublets,  are  not 
significantly  different  for  the  comparison  of  (A)  lcfr-accurate  (252  to  274  trials)  with  (B)  left-inaccurate 
(254  to  284  trials)  condinons,  or  (Ci  right-accurate  (291  to  309  trials)  with  (D)  nghr-inaccuratc  (282 
to  304  trials  i  condinons. 


Fig.  3.  (A)  Averaged,  unfiltered  vemcal 
and  horizontal  eye-movement  signals  dur¬ 
ing  the  cue-to-snmulus  period  for  nght- 
and  left-hand  trials  from  the  seven  sub¬ 
jects.  (B)  Averaged  muscle  potential  sig¬ 
nals  from  right  flexor  digiton  during  the 
cue-to-snmulus  period  (arbitrary  units). 
(C)  Muscle  potentials  dunng  oven  move¬ 
ments  were  mam'  times  larger.  (Averaged 
muscle  potennal  signals  from  left  flexor 
digiton  showed  the  same  rclanon.)  Since 
there  is  no  evidence  of  eve  or  muscle 
movements  during  either  right-  or  left- 
-o  75  -o  s  -0.25  o  0.25  o  s  hand  preparation.  the  covariance  patterns 
(997  Tnals)  Time  (see)  in  Fig  4  are  not  due  to  oven  movements. 


accurate  right-hand  perrormance  involved 
predominantly  left  hemisphere  sites,  partic¬ 
ularly  left  frontal,  central,  parietal,  and  an- 
tcro-parietal  sites  i23i.  .All  24  significant 
covariances  involved  sites  on  the  left  side, 
and  18  . 75%  i  of  these  were  exclusively  on 
the  left  side.  The  covanance  partem  preced¬ 
ing  subsequently  accurate  left-hand  per¬ 
formance  for  this  interval  involved  predomi- 
nandv  right-hemisphere  sites.  Of  18  signifi¬ 
cant  covariances  in  this  partem.  13  (72%  i 
involved  ngnt  hemispheric  sites.  The  neht- 
sided  central,  parietal,  and  antcro-panetal 
sites  were  most  prominent,  compared  with 
corresponding  prominent  contralateral  sites 
for  the  nght-hand  accurate  partem.  The  left 
frontal  site  was  prominent  before  both  Icfr- 
and  ngnt-hand  performance.  The  midline 
central  and  antero-central  sites  were  promi¬ 
nent  in  the  left-hand  partem  but  were  not 
among  the  most  prominent  in  the  nghr- 
hand  pattern. 

Onlv  two  significant  covariances  were  re¬ 
lated  to  subsequendv  inaccurate  nght-hand 
performance  in  this  interval,  namelv.  left 
panctal  with  left  frontal  and  antero-panetal 
with  left  frontal.  In  contrast,  the  subsc- 
quendv  inaccurate  covanance  pane  i  for  the 
left  hand  was  more  bilateral  and  complex 
than  the  subscauendv  accurate  partem.  The 
panems  for  the  rwo  accuracy  condinons 
differed  both  in  scale  and  pattern  for  the  left 
hand,  but  onlv  in  scale  for  the  right  hand 
(24).  Unlike  the  betwcen-channel  covan¬ 
ance  patterns,  the  CNV  amplitude  maps 
were  highjv  similar  for  both  accuracy  condi¬ 
tions  and  hands  and  were  not  useful  in 
dctc.mimng  what  areas  would  covan' . 

Through  the  use  of  stansneal  partem  clas- 
sificanon  procedures,  covanances  shown  in 
Fig.  4  were  considered  possible  vanables  to 
distinguish  subsequent  performance  accura¬ 
cy.  The  tnals  of  each  of  the  seven  sublets 
were  classified  by  equanons  developed  on 
the  tnals  of  the  other  six  subjects  (25).  The 
overall  discnrrunauon  of  subsequendv  accu¬ 
rate  from  subsequendv  inaccurate  tnals  was 
59%  i T5  <  0.0 1 ;  for  right-hand  and  57% 
(P  <  0.01 1  for  left-hand  performance.  Dis¬ 
crimination  of  subsequent  nght-hand  per¬ 
formance  accuracy  was  above  57%  for  six 
subjects  but  was  50%  for  the  seventh.  For 
left-hand  performance,  discrimination  for 
three  subtects  ranged  from  56%  to  67%, 
and  was  53%  or  below  for  four  subjects 
(who  had  fewer  trials  overall).  Average  clas¬ 
sification  of  each  fifth  of  the  tnals  from  the 
four  subjects  with  lowest  left-hand  discrimi- 
nanon,  calculated  from  equanons  developed 
from  the  other  four- fifths,  was  61% 
(P  <  0.001).  This  suggests  that  the  four 
subjects  had  similar  covanance  patterns  be¬ 
fore  left-hand  performance,  which  differed 
from  those  of  the  other  three  subjects.  The 
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Fig.  4.  View  or  :hc  iigniricar.t  P  <  0.05 1  be- 
rween-viunnci  CNV  covariance  pancms.  looking 
down  it  the  top  or  cne  head,  -.upenmposed  on 
maps  ot’CNV  ampnrude.  Measurements  arc  from 
an  interval  500  to  875  msec  alter  the  cue  for 
subsequently  accurate  and  inaccurate  lert-hand 
(topi  and  right-hand  bottomi  visuomocor  task 
Denormance  bv  seven  right-handed  men.  The 
thickness  or  a  covariance  line  is  proportional  to 
the  negative  logantnm  or  its  signuicancc  i  from 
0.05  to  0.001).  A  violet  lute  indicates  the  covari¬ 
ance  is  positive,  whiie  a  blue  line  is  negative. 
Covariances  involving  lefr  frontal  and  appropri¬ 
ately  contralateral  central  and  parietal  electrode 
sites  are  prominent  in  patterns  for  subsequently 
accurate  performance  of  both  hands.  The  magni¬ 
tude  and  number  of  covariances  are  greater  before 
subsequently  inaccurate  lefr-hand  performance  bv 
these  ngnt-handed  subtects  and  are  more  widely 
distributed  tnan  the  left-hand  accurate  pattern. 
For  the  ngnt  hand,  few  er  and  a  enter  covariances 
characterize  subsequently  inaccurate  performance. 
The  amplitude  maps  are  similar  for  the  four 
conditions  and  do  not  indicate  anv  of  the  speciric 
differences  evident  m  the  covariance  patterns. 


greater  uniformin’  for  right-  over  left-hand 
discrimination  suggests  tnat  there  are  similar 
covariance  patterns  among  the  strongly 
ngnt-handed  subiects  preceding  accurate 
and  inaccurate  right-hand  performance,  and 
a  divergence  of  patterns  preceding  left-hand 
performance.  Although  there  were  differ¬ 
ences  in  disenminative  power  between  indi¬ 
viduals,  overall  the  group  preparation  pat¬ 
terns  were  effective  in  deciding  an  individ¬ 
ual’s  subsequent  performance  accuracy.  For 
the  one  subiect  with  the  most  trials,  an 
average  classification  of  68%  iP  <  0.001) 
for  subsequent  right-hand  and  62% 
l P  <  0.01)  for  subsequent  left-hand  per¬ 
formance  was  achieved  bv  testing  a  separate 
equation  on  each  firth  of  his  trials,  formed 
from  the  other  four-fifths. 

AJthougn  the  origin  of  these  event-relat¬ 
ed,  bcrween-channe!  covariance  patterns  of 
preparatory  sets  :s  unknown  26 1,  our  re¬ 
sults  suggest  that  preparation  for  accurate 
performance  in  a  visuomotor  task  involves 
several  brain  components  i27):  i  cognitive 
component  manifested  by  invariant  activity 
at  the  left  frontal  covariance  site,  a  hand- 
specific  somesthenc- motor  component  man¬ 
ifested  bv  the  contralateral  central  and  pari¬ 
etal  sites,  and  an  integrative  motor  compo¬ 
nent  manifested  bv  activity  at  the  midline 
central  and  anccro-centrai  sites.  The  last 
component  was  strong  in  the  pattern  pre¬ 
ceding  accurate  left-hand  performance  and 
weaker  in  the  pattern  preceding  accurate 
right-hand  performance.  For  both  hands, 
preparatory  covariance  patterns  were  differ¬ 
ent  from  those  accompanying  acruai  re¬ 
sponse  execution.  Covariance  patterns  pre¬ 
ceding  inaccurate  performance  bv  each  hand 
differed  markedly.  The  relative  lack  of  signif¬ 
icant  covariances  preceding  inaccurate  right- 


LEFT  PREPARATION 

uv / cm2  ACCURATE 


0.072 


8 

i  "  « 


-0.153 


pLJ| 


.05  10~2  10“  3  1  Pos*  Ne9* 

Significance  C0VAR|ANCE  *9" 


^  A  FtlUr 
500-875 
mstc 
Post— Cu* 

EECSL 


RIGHT  PREPflRfflTON 


uV/cm2 

0.061 


-0.153 


mm 


.05  ia-2  io-3 
Significance 


2W.'-  .1 


COVARIANCE 


Pos.  Neg. 
Sign 


A  nit*' 
500-875 

m3«c 
Post— Cu« 

EECSL 


hand  performance  mav  be  interpreted  as 
evidence  for  a  weakened  preparatory  set.  By 
contrast,  the  complex,  anatomically  diffuse 
but  strong  patterns  in  the  left-hand  condi¬ 
tion  suggest  chat  inaccurate  performance  bv 
the  nondormnant  hand  of  strongly  right- 
handed  subjects  mav  result  from  erroneous, 
possiblv  confounded,  preparatory  sets. 

Our  evidence  for  distributed,  coordinated 
preparatory  components  of  human  visuo¬ 
motor  performance  is  consistent  with  earlier 
studies  of  this  behavior.  The  involvement  of 
the  lefr  froncai  site  is  consistent  with  evi¬ 
dence  that  preparatory  sets  in  humans  are 
synthesized  and  integrated  in  iefr  dorsolater¬ 
al  prefrontal  cortices  ‘4.  5'i.  The  finding  of 


an  appropriately  latcralizcd  pancro-centrai 
somesthenc- mocor  component  is  consistent 
with  data  that  show  neuronal  tiring  patterns 
before  motor  responses  m  the  motor  cortex 
of  nonhuman  primates  and  localized  poten- 
nais  in  the  somestheue  cortex  of  humans  at 
the  same  time  iL  25).  Finally,  a  midline 
antero-central  tntegranve  motor  component 
is  consistent  with  known  involvement  of 
premotor  and  supplementary  motor  regions 
in  imnanng  existing  motor  schemes  and 
establishing  new  ones  ij.  29V 
Our  results  demonstrate  thar  the  human 
brain,  unlike  a  fixed- program  computer, 
dynamically  "programs"'  its  distributed,  spe¬ 
cialized  subsystems  in  anticipation  of  the 
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need  to  process  certain  tvpes  of  information 
and  take  certain  typos  of  action.  VVhen  these 
preparatory  sets  are  incomplete  or  incorrect, 
subsequent  performance  is  likely  to  be  inac¬ 
curate. 
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Lassen,  ibid  50.  426  1980:,  P  E.  Roland,  B. 
Larsen.  N.  A.  Lassen.  E.  Slunhoi./.  Seurophynol.  43, 
1 18  ( 1980).  The  major  pea*  in  the  Lapiaaan  wave¬ 
form  during  the  response  was  centered  62  msec  after 
response  onset  and  was  approximately  190  msec 
wide;  therefore,  a  4-  to  ~-Rz  bandpass  filter  and  a 
covariance  interval  of  187  msec  were  used.  Given 
the  high  signal  strength  of  oven  movement  and  the 
resulting  large  number  of  significant  response- relat¬ 
ed  covariances,  onlv  the  top  standard  deviation  of 
significant  covariances  could  be  showm  in  rig  1 
without  creating  an  ovcriv  complex  display .  If  ail 
significant  covariances  are  considered,  left-sided  co¬ 
variances  were  significants  greater  than  the  compa¬ 
rable  nght -sided  ones  bv  the  [r(78t  =  18.5, 
P  <  0.000 1  ]  for  the  nght-hand  response.  Signifi¬ 
cant  nght -sided  covin  an  ccs  were  significantly  great¬ 
er  dun  tne  left-sided  ones  (r(78)  *  21.5, 
P<  0.001]  for  the  lefr-rund  response.  The  nght 
amenxentral  electrode  site  did  not  appear  in  die 
left-hand  pattern  m  Fig.  1  because  its  significance 
was  shghtiv  below  the  1-SD  cutoff. 

20.  We  made  amplitude  maps  bv  determining  the  aver¬ 
age  amplitude  of  the  Lapiaaan  waveform  for  each 
electrode  site  over  the  same  interval  used  to  calculate 
covariances  and  interpolating  between  sites.  Ampli¬ 
tude  was  represented  on  a  color  scale,  with  red 
representing  the  maximum  and  violet  the  minimum 

21.  A  delta  filter  (low-pass  cutoff  at  3  Hz)  and  a 
covanancc  interval  width  of  375  msec  were  used  to 
studv  the  low-frequene-  CNV  component. 

22.  Comparison  bv  tne  :  rest  of  mean -squared  ampli¬ 
tude,  measured  on  each  Laplacian  waveform  over 
the  same  500-  to  8 75 -msec -centered  postcuc  interval 
as  was  used  for  the  between  crunne I  covanancc) 
between  subsequently  accurate  and  inaccurate  con¬ 
ditions  of  cacn  hand,  was  not  significant  at 


P  <  0  05  Similarity  between  the  sets  of  CNV  am 
piitudcs.  or  tne  two  covanancc  maps,  was  measured 
with  an  estimate  of  tne  corrciaoon  and  its  confi¬ 
dence  interval  Correlation  was  not  used  m  this 
context  to  assess  linear  dependence.  For  the  small 
number  of  repeated  measures,  a  normal  discnounon 
could  not  be  confirmed.  Therciorc.  robust,  resistant 
estimates  were  calculated  with  a  distnbuaon-inde- 
pendent  “bootstrap''  Monte  Carlo  procedure  [B. 
Efron.  The  Jaejmite.  The  Bootstrap .  and  Other  Resam¬ 
pling  Plans  (Society  for  Industrial  and  Applied 
Mathematics.  Philadelphia,  1982)],  which  generates 
an  cnsembic  of  conreiaoon  vaJues  from  randomly 
selected  choices  of  the  repeated  measures.  When  the 
distributions  of  CNV  amplitudes  from  subsequently 
accurate  and  inaccurate  conditions  were  compared 
according  to  this  procedure,  the  correlations  were 
0.84  *  0.16  before  nghr-nand  performance  and 
0.83  r  0.14  before  left-hand  performance  Thus,  it 
is  not  possible  to  discriminate  between  subsequently 
accurate  and  inaccurate  waveforms  on  tne  basis  of 
the  mean -squared  amplitude  of  tne  averaged  event  - 
related  CNV’  waveforms 

23.  Tncse  four  sites  were  the  most  prominent  in  that  the 
numbers  of  significant  covariances  in  which  tnev 
were  involved  each  exceeded  half  of  the  maximum 
number  of  anv  site  rune  for  the  antero- panetal  site. 
The  otner  sites  in  the  pacrem  were  involved  in  one- 
third  or  fewer  of  mat  maximum.  The  same  entenon 
was  used  to  ludge  wnich  sites  were  most  prominent 
in  the  left-hand  partem. 

24.  In  addition  to  me  cue -to- sum  ulus  penod.  the  post- 
snmuius  and  posncedback  penods  also  differed  in 
accuracy,  but  me  response  penod  did  not  1 18 ..  The 
signal  strength  of  p  res  um  uius  covin  an  ccs  was  much 
smaller  than  those  dunng  oven  responses,  that  is. 
the  scale  of  significance  was  tnree  orders  of  magni¬ 
tude  smaller  The  smaller  number  of  pres  am  ulus 
covananccs  allowed  all  significant  covananccs  to  be 
shown  in  Fig.  4  Com  pan  son  bv  r  test  of  me  sets  of 
subsequently  accurate  and  inaccurarc  covananccs 
was  significant  at  P<  0.001  for  both  iefit- 
|r(38)  *  5.57]  and  nght-hand  f?(23)  =  7.701  com¬ 
parisons.  The  bootstrap  correiaaon  (22)  between 
covanancc  partems  before  subsequently  accurate 
and  inaccurate  performance  from  channel  pairs  that 
war  jiemficanr  for  either  condmon  was  0.57  r 
0  09  for  the  nght  hand  and  0.10  r  0.14  for  me  lcfr 
hand.  The  ;  test  and  bootstrap  correiaaon  results, 
taken  together,  suggest  that  me  left-hand  accurate 
and  inaccurate  condioons  differ  both  in  scale  and  in 
pattern,  whereas  ngnt-hand  results  differ  oniv  in 
scale. 

25.  The  classifier  was  a  nonlinear,  rwo-lavercd.  adapeve 
deasion  network  • :  15 ) ,  A  Gevins  et  al..  Science  203. 
665  1 1979);  A.  Gevins  et  al..  Psychophysiology  22.  32 
(1985  )1  that  decided  whether  subsequent  perform¬ 
ance  was  accurate  or  inaccurate  from  CNV -interval, 
between -cnannel  covan ances  of  each  trial.  This  aigo- 
nthm  produced,  bv  a  recursive  procedure,  ciassifica- 
aon  cquaaons  consisting  of  weighted  combmaaons 
of  me  decisions  of  discriminant  functions,  wnich 
themselves  consisted  of  weighted  comoinanons  of  a 
subset  of  the  covanancc  values  of  Fig  4  Cross- 
validation  of  the  cquaaons  was  performed  bv  testing 
cquaaons  on  data  that  were  not  used  ro  derive  them. 
Significance  was  determined  according  to  the  bino¬ 
mial  distnbuQon.  Details  of  the  appiicaaon  of  pat¬ 
tern  dassificanon  procedures  to  tne  andvsis  of  brain 
signals  have  been  nresented  [A.  Gevins.  IEEE  Trans 
Pattern  Anal  Mach  Intel! .  2,  282  (1980).  in  (J4), 

chap.  1".  _  and  N  Morgan,  IEEE  Trans. 

Burned  Ena.,  in  press].  These  conservaavc  proce¬ 
dures  norwitnstanding.  we  must  caunon  that  the 
degree  or  generalization  of  these  results  to  the 
oopulanon  at  large  is  unknown  and  can  be  deter¬ 
mined  oniv  bv  addroonai  srudics  with  new  subjects. 

26.  Previous  studies  that  measured  mrerciectrode  corre- 
lanon  have  suggested  a  coracaj  origin  tor  these 
patterns  il3-ip)  The  difficult  problem  of  identify¬ 
ing  the  generators  of  these  patterns  is  the  focus  of 
current  work.  At  present,  the  focal,  spanaiiv  separat¬ 
ed  patterns  tend  to  rule  out  volume-conducted 
aenvirv  from  one  or  rwo  cortical  or  subcortical 
generators 

27.  D  Ruchkin.  S.  Sutton,  D  Mahaffey.  EEG  Clm 
Seurvphmol  63.  445  ( 1986 ' 

28.  V  B.  Mountcastie.  J  R  Soc.Med  71,  14  <  1978);  B 
Lee.  H.  Ludcrs.  R.  Lesser.  D  Dinner.  H  Moms, 
Ann  .Vnmx  20.  32  1986i 
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ALAN  S.  OEVINS  and  BRIAN  A.  CUTILLO 

EEC  Systems  Laboratory,  1855  Folsom  Street,  San  Francuco,  CA  94103  (U.S.A.) 


11.1.  Introduction 

11.1.1.  Overview 

Brain  potentials  have  been  widely  used  for  studying  brain  activity  associated  with 
higher  mental  functions  in  humans,  either  as  continuous  EEG  or  as  averaged 
event-related  potentials  (ERPs).  Although  over  50  years  old.  human  ‘psycho-neu¬ 
rophysiology’  is  still  in  its  infancy,  awaiting  development  of  more  powerful  tools 
for  measuring  the  minute  signals  of  neurocognitive  processes  and  locating  their 
sources  in  the  brain.  Such  tools  include  recordings  with  many  channels,  event-re¬ 
lated  magnetic  field  (ERMF)  measurements,  and  advanced  signal  processing  tech¬ 
niques.  However,  even  with  the  most  advanced  techniques  currently  available,  the 
best  we  can  hope  to  do  is  observe  some  signs  of  the  timing  and  location  of  mass 
neural  activity  associated  with  cognition.  The  advantages  of  non-invasiveness  and 
fine  temporal  resolution  are  motivating  the  development  of  better  recording  and 
analytic  techniques. 

11.1.2.  EEG  STUDIES  OF  COGNITION 1 

The  first  report  of  an  EEG  phenomenon  associated  with  cognition  was  Hans  Ber¬ 
ger’s  observation  that  the  aipha  rhythm  decreased  in  amplitude  during  mental 
arithmetic  (Berger  1929).  There  have  been  many  subsequent  EEG  studies  of  cog¬ 
nition.  with  recent  emphasis  on  hemispheric  specialization  (reviewed  in  Gevms 
and  Schaffer  1980;  Gevins  1983;  Butler  and  Glass  1986).  The  major  limitation  in 
EEG  studies  is  that  the  long  EEG  segments  required  make  it  impossible  to  resolve 
the  spiit-second  stages  of  cognitive  processing. 
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11.1.3.  EvENT-XELATED  POTENTIALS  (ERPs)  AKD  COGMTnON 


ERPs  are  a  valuable  tool  for  studying  the  mass  neural  activity  generated  during 
cognitive  processes.  They  are  usuallly  studied  by  averaging  a  number  of  brief  EEG 
segments  time-registered  to  a  stimulus  or  response  in  simple  cognitive  tasks.  Ave¬ 
raging  enhances  signals  correlated  in  time  with  the  stimulus  and  subsequent  task 
processing  and  suppresses  unrelated  background  activity.  An  averaged  ERP  wave¬ 
form  consists  of  a  series  of  positive  and  negative  waves.  A  significant  change  in 
latency  or  amplitude  of  a  peak,  between  experimental  conditions  which  differ  in 
a  specific  cognitive  aspect  of  task  performance,  is  assumed  to  reflect  differences 
in  the  mass  neural  activity  associated  with  that  cognitive  process.  Since  the  avera- 


■ 

1 1 

Fig.  1.  The  contingent  negative  vanation  (CNV),  reponed  to  be  related  to  expectation  in  the  interval 
between  warning  and  imperative  stimuli,  from  onginal  report  of  W  G.  Waiter  et  al.  (1964).  Averaged 
waveforms  (from  vertex  electrode  referenced  to  mastoid)  to  unpaired  dick  (A)  and  flash  (B)  stimuli, 
paired  click  and  flash  stimuli  1  sec  apart  with  no  task-relevant  contingency  (C).  and  CNV  in  the  1  sec 
interval  between  the  paired  stimuli  when  they  ire  made  contingent  by  requiring  a  button  press  io  the 
second  stimulus  (D). 
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ged  ERP  is  synchronized  to  a  stimulus  or  response,  it  is  possible  to  make  inferences 
about  the  sequence  and  timing  of  task-associated  processes,  such  as  pre-stimulus 
expectation  or  preparation,  encoding  of  stimulus  features,  evaluation  of  the  'mean¬ 
ing'  of  the  stimulus,  and  response  selection  and  execution.  ERP  components 
which  are  influenced  primarily  by  physical  stimulus  properties  are  called  ‘exoge¬ 
nous,’  while  those  which  covary  mainly  with  cognitive  factors  are  called  ‘endoge¬ 
nous.’  In  actuality,  most  components  are  sensitive  to  both  exogenous  and  endoge¬ 
nous  factors. 

In  the  mid-sixties,  two  separate  teams  of  researchers  discovered  that  certain 
ERP  components  are  related  not  just  to  ‘exogenous’  factors,  such  as  stimulus  prop¬ 
erties  or  motor  activity,  but  also  to  cognitive  (‘endogenous')  factors.  In  1964. 
W.G.  Walter.  Cooper,  Aldridge.  McCailum  and  Winter  reported  an  ERP  event 
related  to  expectation  (Walter  et  ai.  1964).  This  ‘contingent  negative  variation' 
(CNV)  is  a  slowly  increasing  negative  shift  preceding  an  expected  stimulus,  often 
visible  in  the  unaveraged  EEG  (Fig.  1).  The  CNV  was  found  to  vary  during  stages 
of  conditioned  learning,  with  motivation,  attention  and  distraction,  and  other  cog¬ 
nitive  factors  (reviewed  in  Tecce  1972). 

At  almost  the  same  time,  Sutton.  Braren.  Zubin  and  John  discovered  a  positive 
wave  peaking  about  300  msec  after  infrequent  task-relevant  stimuli  (Sutton  et  ai. 
1965;  Fig.  2).  This  ‘P300’  or  ‘P3’  peak  is  considered  endogenous  because  its  scalp 
topography  and  behavior  do  not  depend  on  physical  stimulus  properties  per  se. 
but  on  their  ‘meaning’  in  the  context  of  the  task.  P300  is  evoked  by  many  types 
of  paradigms  (see  Roth  1978;  Tueting  1978;  Donchin  1979;  and  Duncan-Johnson 
and  Donchin  1982  for  reviews),  but  the  most  common  factors  which  influence  its 
behavior  are  stimulus  frequency  and  task  relevance.  An  especially  cogent  Finding 


TIME  (MILLJSECONOSJ 


Fig  2.  Illustration  of  P300  ERP  from  the  original  report  of  Sutton  et  ai.  ( 19P3)  ERP  waveforms  from 
5  persons  elicited  by  infrequent  unexpected  (  uncertain  )  and  frequent  expected  (  certain  )  auditory 
stimuli  P300  is  the  large  positive  wave  peaking  about  300  msec  post  stimulus. 
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SINGLE  SOUND 


milliseconds 


Fig.  3.  Missing  stimulus  ERP  or  'emitted  potential'  elicited  bv  infrequent  omissions  of  the  second 
stimulus  in  regularly  spaced  pairs  of  auditory  stimuli.  The  'missing  stimulus  potential'  is  vis.ble  in  the 
3  lower  waveforms  as  a  large  positive  wave  peaking  about  300  msec  after  the  time  when  the  missing 
stimulus  was  expected  (indicated  by  the  rightmost  mangle).  The  preceding  solid  mangles  indicate  the 
presentation  times  of  actual  auditory  stimuli.  (Adapted  from  the  original  report  of  Sutton  et  al.  I%7 .) 


was  the  elicitation  of  P300  by  the  absence  of  a  stimulus  in  a  regularly  timed  se¬ 
quence  of  stimuli .  This  'missing  stimulus  ERP.'  obtained  by  averaging  the  brain  poten¬ 
tials  registered  to  the  time  the  missing  stimuli  should  have  occurred,  lacks  eariy 
exogenous  peaks  such  as  N100  and  P200.  but  contains  P300  and  other  endogenous 
waves  (Sutton  et  al.  1963.  1967;  Ruchkin  et  al.  1975)  (Fig.  3). 


11.2.  Basic  assumptions  of  the  ERP  method 

11.2.1.  COCNITTON  CAN  BE  MEASURED 

11.2.1.1.  Cognitive  faaors  can  be  isolated 

Mental  events  cannot  be  measured  directly;  we  can  only  observe  behavioral  and 
neural  activitv  associated  with  them.  Psychological  studies  use  behavioral  measures 
such  as  reaction  time  to  define  component  stages  of  cognitive  processing  and  make 
inferences  about  their  nature  and  timing.  The  validity  of  any  cognitive  study  de¬ 
pends  on  the  effectiveness  of  experimental  design  and  controls  in  manipulating  a 
specific  cognitive  factor  while  holding  all  other  factors  constant.  For  example.  3 
typical  memory  task  involves  not  only  memory,  but  also  expectation  and  prepa¬ 
ration.  encoding  of  stimulus  features,  comparison  wuh  memory,  and  selection  and 
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execution  of  a  response.  The  study  of  brain  signals  associated  with  cognition  is 
further  complicated  by  their  more  complex  sensitivity  to  physical  aspects  of  stimuli 
and  response  movements. 

11.2.1.2.  The  timing  of  neurocognmve  processes  is  consistent 

The  averaged  ERP  method  assumes  that  task-related  cognitive  processes  do  not 
vary  much  in  timing  from  trial  to  trial.  If  neurocogmtive  functions  were  truly  'hard 
wired.'  this  might  be  a  reasonable  assumption,  but  there  is  little  evidence  to  sup¬ 
port  such  an  idea.  Trial-to-mal  variations  in  the  timing  of  neurocogmtive  processes 
affect  both  the  latency  and  the  amplitude  of  an  ERP  component,  since  latency  and 
the  amplitude  in  the  averaged  waveform  are  interd  .pendent.  Measures  of  the 
integrated  amplitude  ('area')  of  a  wave  are  sometimes  used  to  alleviate  the  exces¬ 
sive  sensitivity  of  peak  amplitude  measures  to  these  variations,  and  latency  correc¬ 
tion  for  particular  peaks  has  been  attempted  by  adjusting  the  single-trial  data  in 
time  (Woody  1967;  Aunon  1978;  see  Section  11.6.1  in  this  chapter,  and  Chapter 
5  in  Vol.  1).  However,  without  careful  filtering  (Aunon  et  al.  1981).  the  amplitude 
increase  produced  by  latency  correction  procedures  may  be  influenced  by  unrela¬ 
ted  EEG  activity,  usually  an  alpha  frequency  component  (Chase  et  al.  1984).  The 
disappearance  of  an  experimental  effect  after  latency  correction  has  been  reported 
(Kutas  et  al.  1977).  suggesting  that  the  amplitude  results  of  other  studies  could  be 
due  to  differences  in  variance  of  the  timing  of  task-related  signals  between  condi¬ 
tions. 

Tne  measurement  of  peak  latency  is  another  important  consideration,  especially 
in  chronometric  studies  (see  Sections  11.4.2.2-3  and  11.5.1-2.  below).  Latency  is 
measured  with  respect  to  a  peak  chosen  either  by  visual  inspection  or  automated 
programs  which  choose  the  point  of  largest  amplitude  within  a  (usually  broad) 
latency  range.  However,  the  presence  of  high  frequency  activity  can  cause  spurious 
variability  in  latencies  determined  by  automated  programs,  especially  for  broad 
peaks,  due  to  tiny  'pcaklets'  which  are  clearly  not  the  center  of  the  wave.  In  this 
case  low-pass  filtering,  or  measuring  the  latency  as  the  center  of  gravity  of  the 
wave  may  be  appropriate.  Even  with  visual  choice  of  peaks,  the  close  overlap  of 
two  separate  peaks  of  slightly  different  distribution  can  cause  confusion.  For  a 
discussion  of  the  methods  of  latency  .easurement  see  Callaway  et  al.  (1983). 

11.2.1.3.  Learning  and  automatization  are  not  appreciable 

The  issues  of  learning,  automatization  and  habituation  must  be  considered  when 
stimuli  and  tasks  are  repeated  hundreds  of  times.  For  example,  in  one  study  a 
reduction  in  P300  amplitude  was  attributed  to  fatigue  after  12  h  ot  continuous  task 
pertormance  i  Mane  et  ai.  1983).  In  such  a  case,  follow-up  recordings  in  an  unfa- 
tigued  state  should  be  made  to  determine  whether  the  effect  was  actually  due  to 
fatigue  rather  than  automatization,  since  reduction  of  P300  amplitude  with  exten¬ 
sive  task  performance  has  been  reported  (Kramer  et  al.  1983;  Woods  and  Cour- 
chesne  1983).  Signs  of  habituation  have  been  observed  in  many  ERP  components 
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(Callaway  1973;  Megcla  and  Tcyler  1979),  including  the  CNV  (reviewed  in  Love¬ 
less  1979)  and  the  P300  (Courchesne  1978;  reviewed  in  Roth  1973  and  Verbaten 
1983). 

11.2.2.  Overlapping  components  can  be  resolved 

The  averaged  ERP  waveform  is  assumed  to  be  a  linear  composite  of  spatially  and 
temporarily  overlapping  'components’  generated  in  distinct  neural  systems.  How¬ 
ever.  it  is  naive  to  consider  that  peaks  in  the  averaged  ERP  are  distinct  components, 
and  precise  measurement  of  the  overlapping  components  in  the  composite  wave¬ 
form  is  problematic  (see  Discussion  in  Callaway  et  al.  1978).  In  some  cases,  rela¬ 
tively  simple  experimental  or  analytic  methods  may  be  effective  in  distinguishing 
separate  components.  One  example  is  the  'selective  attention  effect’  initially  attri¬ 
buted  to  the  N100  peak  (Hillvard  et  al.  1973).  The  amplitude  of  N100  was  observed 
to  be  larger  for  ’attended’  stimuli,  which  were  selected  on  the  basis  of  simple 
physical  properties,  than  for  those  which  were  ‘ignored.’  However,  it  was  subse¬ 
quently  shown,  by  manipulation  of  inter-stimulus  intervals  and  by  subtracting  the 
ERP  for  ’ignored’  stimuli  from  the  ERP  for  'attended'  stimuli,  that  the  amplitude 
increase  was  actually  due  to  a  broad,  low-amplitude  negative  wave  (‘processing 
negativity’)  which  overlapped  the  N100  peak  (Hansen  and  Hillvard  198C;  Naatanen 
et  al.  1978)  (Fig.  4).  Another  example  is  the  clarification  of  the  relationship  of 
P300  latency  to  reaction  time  in  a  study  by  Friedman  et  al.  (1978).  Some  studies 


Fig  -t  The  N100  selective  attention  effect':  an  example  of  overlacping  event-related  waves.  The 
apparent  amplitude  increase  of  the  NIOO  peak  :n  the  averaged  ERP  for  'attended'  stimuli  is  actually 
due  :o  a  broad,  low  amplitude  negative  wave  CNd'  or  processing  negativity')  which  overlaps  N100. 
This  was  demonstrated  bv  manipulating  the  length  of  the  mter-stimulus  interval,  and  by  subtracting 
the  ERP  for  unattended'  from  the  ERP  for  attended'  stimuli.  The  illustration  shows  the  subtraction 
ERPs  for  inter-stimulus  intervals  of  250  msec  (dasned  line).  S00  msec  (solid  line)  and  2000  msec  (dotted 
line),  i  Adapted  from  Naatanen  et  al.  1982.) 
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reported  moderate  correlations  of  P300  latency  and  reaction  time,  while  the  results 
of  other  studies  were  negative  (reviewed  in  Donchin  et  al.  1978).  By  dividing  their 
data  into  reaction  time  quaniles.  Friedman  and  colleagues  distinguished  4  overlap¬ 
ping  peaks  in  the  P300  range,  only  one  of  which  was  related  to  reaction  time  (Fig. 
5),  suggesting  that  this  might  account  for  the  inconsistent  findings  of  other  studies. 

It  is  not  often  possible  to  distinguish  overlapping  components  by  such  simple 
methods.  As  a  more  general  solution  to  this  problem,  it  was  proposed  that  indepen¬ 
dent  components  could  be  mathematically  defined  by  principal  components  analy¬ 
sis  PC  A ;  see  Donchin  and  Heffley  1978).  This  method  was  used  to  demonstrate 
the  .ndependence  of  P300  and  CNV  (Donchin  et  al.  1975).  and  has  become  a  fairly 
common  practice  in  ERP  studies.  PCA  determines  statistically  independent  factors 
representing  different  sources  of  variance  in  the  data  by  forming  linear  combina¬ 
tions  of  the  ERP  amplitude  values  at  each  sampled  timepoint.  These  factors  are 
maximally  uncorrelated  with  each  other  and  account  for  most  of  the  variance  in 
the  data  (see  Chapters  3.  5,  16  and  17  in  Vol.  1).  Usually.  4-10  factors  will  account 
for  more  than  90%  of  the  variance,  depending  on  the  complexity  of  the  sources 
of  variance  (Donchin  and  Heffley  1978;  Kavanagh  et  al.  1978).  In  spite  of  its 
popularity,  the  PCA  method  has  inherent  limitations.  In  particular,  it  is  difficult 
to  determine  which  experimental  variable  or  ‘source  of  variance'  a  particular  factor 
reflects  (Rosier  and  Manzey  1981;  Wood  and  McCarthy  1984).  Other  difficulties 
and  possible  alternatives  to  PCA  are  discussed  in  Section  11.6.2  (below),  and  Voi. 
1.  Chapter  14. 


Fig  5.  Separation  of  overlapping  waves  in  ihe  averaged  ERP  in  a  visual  discrimination  task  by  comput¬ 
ing  subaverages  of  mats  according  to  reaction-time  quaniles.  Venicai  lines  in  the  parietal  iPl)  channel 
mark  mean  reaction  times  (RT).  time  scale  is  110  msec  per  division,  and  arrow  marks  stimulus  onset. 
Comoanson  of  central  (Cz)  and  panetal  (Pz)  waveforms  from  shon  to  long  RT  quaniles  reveals  that 
the  late  positive  complex  is  composed  of  several  overlapping  peaks.  Since  only  the  longest-iatencv  peak 
is  correlated  with  RT  (r» 0  991.  it  becomes  clearly  distinct  (at  about  52£  msec)  in  the  longest  RT 
quamie.  (Adapted  from  Friedman  et  al.  1978.) 
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11-3.  Methodological  requirements  of  neurocognitive  studies 

Methodological  requirements  for  neurocognitive  studies  have  been  discussed  in 
several  books  and  review  articles  (Donchin  et  ai.  1977;  Thatcher  and  John  1977; 
Gevins  and  Schaffer  1980;  Gevins  et  al.  1985).  As  in  any  experiment,  it  is  essential 
to  control  all  factors  not  directly  related  to  the  intended  experimental  manipulation. 
Special  attention  to  experimental  control  is  required  in  neurocognitive  studies, 
since  brain  signals  are  very  sensitive  to  many  factors,  and  neurocognitive  signals 
are  small  in  relation  to  'background'  activity.  Only  a  few  of  the  more  important 
issues  are  discussed  below. 


11.3.1.  Elimination  of  artifacts 

Contamination  of  brain  signals  by  non-neural  potentials  is  a  serious  problem.  The 
major  sources  of  physiological  artifacts  are  eye  movements  and  blinks,  and  muscle 
activity,  particularly  of  the  scalp  muscles.  Artifacted  data  is  usually  eliminated  by- 
visual  inspection  of  polygraph  tracings  or  automated  algorithms  which  reject  trials 
exceeding  preset  voltage  levels.  However,  small  undetected  eye  movements  or 
muscle  activity  which  may  be  related  in  time  to  the  stimulus  or  response  will 
summate  in  averages  and  could  cause  spurious  results.  It  is  possible  that  task-re¬ 
lated  eye  and  muscle  activity,  however  slight,  accompanies  all  cognitive  processing. 
Lateral  eye  movements  (saccades)  and  blinks  have  been  shown  to  be  correlated 
with  cognitive  processes  (Just  and  Carpenter  1976;  Stem  et  al.  1984).  and  activity 
of  the  facial  and  scalp  muscles  has  been  used  to  distinguish  different  cognitive 
tasks  (Cacioppo  et  al.  1983).  Other  sources  of  artifactuai  muscle  potentials  include 
response  movements  and  subvocaiization.  Digital  algorithms  for  removing  the  ef¬ 
fects  of  eye  movements  have  been  developed  for  averaged  ERPs  (Verleger  et  al. 
1982;  Gratton  et  al.  1983).  and  single-trial  data  (Gasser  et  al.  1983;  Bonham  et  al. 
unpublished  manuscript).  They  seem  effective  in  removing  task-related  artifacts, 
and  further  evaluation  of  their  performance,  as  well  as  extension  of  such  methods 
to  muscle  potential  contaminants,  it  eagerly  awaited.  (See  also  Chapter  1  of  this 
volume.)  Contamination  by  physiologic  artifacts  is  also  a  serious  problem  in  brain 
magnetic  field  recordings  and  has  not  yet  been  systematically  studied. 

11.3.2.  Control  of  physiologic  and  experimental  factors 
11.3.2.1.  Arousal 

One  aspect  of  experimental  control  concerns  general  physiologic  factors.  Brain 
signals  vary  with  age.  handedness,  gender,  level  of  autonomic  arousal,  fatigue,  and 
use  of  caffeine,  nicotine,  alcohol  and  various  drugs.  Autonomic  arousal  is  a  special 
problem  which  has  not  received  adequate  attention.  Indirect  indices,  such  as  pupil 
size,  have  been  used  to  measure  the  phasic  arousal  of  the  central  nervous  system 
during  cognitive  tasks  (Kahneman  1973).  More  direct  measures  of  CNS  'activa- 
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tion.’  such  as  EEG  spectral  intensity,  have  been  used  to  measure  gross  differences 
in  global  arousal  (Loomis  et  al.  1938;  Daniel  1966;  Rechtschaffen  and  Kales  1973; 
Simon  et  al.  1977).  and  there  is  no  fundamental  obstacle  to  an  EEG  index  of  phasic 
‘activation'  (Gevtns  et  al.  1977).  However,  it  is  more  difficult  to  distinguish  effects 
associated  with  specific  ‘information  processing'  from  those  which  reflect  the  over¬ 
all  level  of  activity  of  the  central  nervous  system  (Khachaturian  and  Gluck  1969; 
Khachaturian  et  al.  1973).  These  two  types  of  activity  are  not  totally  distinct,  but 
are  aspects  of  the  integrated  functioning  of  the  brain.  Until  technological  advances 
provice  better  means  of  distinguishing  the  two,  control  of  these  factors  must  be 
attempted  by  manipulation  of  task  difficulty,  motivation  and  general  physiological 
factors. 

11.3.2.2.  Isolating  cognitive  processes 

The  essential  issue  of  experimental  design  and  control  bears  on  the  validity  of 
assuming  that  the  experimental  conditions  actually  manipulate  a  specific  neurocog- 
nmve  process,  and  that  the  results  are  not  due  to  factors  not  directly  related  to 
that  process.  Tasks  should  be  simple  and  short  to  reduce  the  possibility  of  trial-to- 
trial  variation  in  performance  and  'strategy.'  but  challenging  enough  to  engage  full 
attention.  The  conditions  of  an  experiment  should  be  controlled  for  irrelevant 
stimulus,  physiological-state  and  response-related  factors  by  experimental  design 
and  a  posteriori  balancing  of  data  sets.  Such  factors  include  the  physical  properties, 
frequency  and  ordering  of  stimuli,  task  difficulty,  automatization,  and  the  timing, 
force,  duration,  hand  and  nature  of  response  movements  (Gevms  et  al.  1979a.  b: 
1980,  1981).  Task  difficulty  should  be  adjusted  according  to  changes  in  perfor¬ 
mance.  as  indexed  by  behavioral  measures  such  as  reaction  time  and  error.  Behav¬ 
ioral  measures  should  be  made  on  each  triai  to  verify  that  the  task  is  properly  per¬ 
formed.  and  to  form  balanced  sets  of  trials  differing  only  in  the  variable(s)  under 
study  prior  to  hypothesis  testing  (Gevins  and  Schaffer  1980;  Gevins  1983;  Gevins 
et  al.  1985.  1986).  If  ERP  peak  amplitudes  are  measured  with  respect  to  a  pre¬ 
stimulus  baseline,  the  effects  of  expectancy  must  be  considered,  especially  when 
inter-stimulus  intervals  are  constant. 

Finally,  the  hypotheses,  paradigms,  and  electrode  locations  should  be  de-igned 
with  careful  consideration  of  the  information  about  localized  neurocognmve  func¬ 
tions  obtained  from  clinical  observations,  metabolic  scanning  techniques  (regional 
blood  flow  and  positron  emission  tomography),  intracranial  recording  and  stimula¬ 
tion  studies,  and  primate  models. 
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11.4.  ERP  events  associated  with  cognition:  two  major  lines  of  research 

11.4.1.  Signals  of  preparation:  the  contingent  negattve  variation  and  the 
readiness  potential 

11.4.1.1.  Cognitive  factors 

The  contingent  negative  variation  (CNV)  is  a  slowly  increasing  negative  potential 
which  occurs  between  two  stimuli  in  tasks  involving  a  contingency  or  association 
of  paired  stimuli  (see  Fig.  i).  It  is  best  seen  when  the  stimuli  are  separated  bv  a 
regular  interval  with  a  response  required  after  the  second  stimulus.  About  ZOO 
msec  after  the  second  stimulus,  there  is  a  sharp  positive  deflection,  termed  the 
‘CNV  resolution.'  The  psychological  conditions  which  reliably  elicit  the  CNV  are 
of  4  general  types  (Hillvard  1973):  (1)  holding  a  motor  response  ready.  (Z)  prepa¬ 
ration  for  a  perceptual  judgment.  (3)  anticipation  of  reinforcement  or  feedback, 
and  (4)  preparation  for  a  cognitive  decision.  The  CNV  was  initially  considered  a 
signal  of  'expectancy'  (Walter  1964).  but  subsequent  studies  demonstrated  that  it 
is  affected  by  many  cognitive  factors  including  attention,  distraction,  preparation 
for  cognitive  tasks,  anticipation  of  information,  ambiguity  and  task  difficulty,  and 
also  by  non-cognitive  factors  such  as  stimulus  intensity,  modality  and  timing,  reac¬ 
tion  time  and  magnitude  of  movement,  motivation,  positive  or  negative  reinforce¬ 
ment.  noxious  stimuli,  arousal,  age.  fatigue,  various  types  of  drugs,  and  clinical 
disorders  including  learning  disability,  senility,  neurosis,  anhedonia.  depression 
and  schizophrenia  (reviewed  in  Tecce  1972;  Rockstroh  et  al.  1982;  Rohrbaugh  and 
Gaillard  1983). 

The  theory  that  the  CNV  reflects  preparatory  processes  assumes  that  the  nega¬ 
tive  potential  shift  reflects  a  build-up  of  'cerebral  potentiality'  which  is  released 
during  the  positive-going  resolution  following  the  second  stimulus  (Walter  1964: 
Rockstroh  et  al.  1982).  The  scalp  distribution  of  the  CNV  is  widespread,  usually 
largest  at  the  midline  central  electrode,  but  also  strong  at  precentral  and  frontal 
sues  (Crow  et  al.  1963).  The  CNV  is  thought  to  reflect  depolarization  of  apical 
dendrites  in  frontal  and  central  cortex  (Waiter  1964).  but  the  theory  that  this 
facilitates  neuronal  action  potentials  has  not  been  proven  (Rebert  1978).  The  pos¬ 
sible  involvement  of  frontal  cortex  is  of  particular  interest,  since  it  is  thought  to 
mediate  higher-order  functions  such  as  preparation  and  planning  (Fuster  1980: 
Goldman-Rakic  1984).  The  theory  that  the  CNV  reflects  task-specific  cortical  acti¬ 
vation  has  been  weakened  by  failure  to  elicit  localization  of  amplitude  with  diffe¬ 
rent  types  of  tasks,  such  as  linguistic,  spatial  and  pattern  discriminations  (Donchin 
et  al.  1973).  or  attention  to  different  sensory  modalities  (see  Hillvard  1973).  Also, 
evidence  for  an  inverse  relationship  of  CNV  amplitude  and  reaction  time,  which 
wouiu  support  the  'potentiality'  hypothesis,  has  been  inconsistent  (reviewed  in 
Tecce  1972).  although  recent  studies  indicate  that  this  may  have  been  due  to  the 
use  of  short  (less  than  2  sec)  intervals  (Brunia  and  Vingerhoets  1980). 
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11.4.1.2.  Sensory  and.  motor  aspecu  of  the  CW 

The  extent  to  which  the  CNV  reflects  cognitive  association  of  two  stimuli  is  still 
unclear.  The  CNV  was  found  to  consist  of  two  overlapping  waves  with  different 
topographies  and  responses  to  experimental  factors  (Low  et  ai.  1966).  These  two 
components  can  be  seen  as  separate  waves  when  the  inter-stimulus  interval  is  2 
sec  or  longer  (Rohrbaugh  et  al.  1976).  The  early  CNV,  sometimes  called  the 
'orienting  wave,’  is  a  frontally  maximum  negative  wave  following  SI,  and  the  late 
CNV,  which  has  been  called  the  'expectancy  wave.'  is  a  centrally  maximum  nega¬ 
tive  shift  preceding  S2.  A  number  of  researchers  believe  that  the  CNV  observed 
at  short  inter-stimulus  intervals  is  merely  a  composite  of  two  separate  negative 
waves  and  is  not  related  to  contingent  association  between  two  stimuli  (Gaillard 
1973:  Rohrbaugh  and  Gaillard  1983).  Rohrbaugh  et  ai.  (1980)  synthesized  a  CNV- 
like  waveform  from  ERPs  accompanying  unassociated  events:  the  slow  negative 
wave  following  unpaired  stimuli  (click  or  flash),  the  readiness  potential  (RP)  pre¬ 
ceding  self-initiated  finger  movements,  and  the  ERP  to  another  unpaired  stimulus 
(Fig.  6).  The  synthetic  waveform  created  by  adding  the  voltage  timesenes  of  these 
3  ERPs.  positioned  in  Lime  to  emulate  a  1  sec  inter-stimulus  interval,  resembled 
the  CNV  recorded  in  a  traditional  paired-stimuli  paradigm. 

The  readiness  potential  (RP)  mentioned  above  is  a  centrally  maximum  negative 
shift  beginning  about  500  msec  before  voluntary  motor  actions  (Kornhuber  and 
Ceecke  1965).  It  responds  to  some  of  the  same  factors  as  the  CNV.  such  as 
motivation  (reviewed  in  McAdam  1973).  but  the  CNV  is  typically  stronger  at 
.  omal  sites  and  is  not  usually  lateralized  (Naatanen  and  Michie  1979b).  whereas 
t  ie  RP  is  larger  over  central  sites  contralateral  to  the  responding  hand  (Deecke 
:nd  Kornhuber  1977).  The  CNV  is  reported  to  be  larger  when  a  motor  response 
is  required  to  the  second  stimulus,  but  also  occurs  in  the  absence  of  response 
reviewed  in  Rockstron  et  al.  1982;  Rohrbaugh  and  Gaillard  1983). 

Many  other  researchers  believe  that  the  negative  waves  associated  with  non-con- 
ungent  events  do  not  fully  account  for  the  behavior  of  the  CNV  and  favor  the  view 
'hat  the  CNV  does  retlect  contingency  to  some  extent  (Doncnin  et  al.  1974;  Ritter 
ct  al.  1984).  The  CNV  has  been  seen  to  develop  when  a  contingency  between  two 
-timuli  is  learned  and  then  decline  when  the  contingency  is  removed.  CNV  ampli- 
ude  is  reduced  by  distracting  stimuli  (Tecce  1973).  instructions  to  change  ’attentio- 
tal  focus-  (McCallum  and  Walter  1968).  and  ’equivocation-  of  contingency  through 
andom  omission  of  the  second  stimulus  (Waiter  et  al.  1964;  Low  et  al.  1966). 
Other  evidence  for  a  non-motor  aspect  comes  from  paradigms  in  which  a  negative 
"lift  is  sustained  through  a  7  sec  sequence  of  contingent  stimuli  (Kutas  and  Hillyard 
.980a.  b.  c),  which  has  not  b-en  observed  for  the  RP.  A  few  studies  have  reported 
topographic  changes  related  to  modality-specific  expectancy,  but  the  effect  is  small 
in  relation  to  overall  CNV  amplitude  (Simson  et  al.  1977a:  Ritter  et  ai.  1980: 
Woods  and  Courcnesne  1983).  All  considered,  it  seems  that  both  the  CNV  and 
the  RP  are  compound  phenomena  which  retlect  neural  activity  related  to  many 
sensory,  cognitive  and  motor  factors. 
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Fig  6  Synthesis  of  CNV-like  waveform  by  superimposing  ERPs  accompanvmg  unassociated  events 
Left:  the  readiness  ootenna!  (RP)  preceding  self-initiated  finger  presses  and  the  ERPs  elicited  bv 
unpaired  clicks  and  flashes  were  positioned  m  time  to  emulate  a  1  sec  inter-stimulus  interval,  and  their 
voltage  time  senes  were  summed  to  produce  the  synthesized  'CNV  '  Right:  comparison  of  synthesized 
(solid  lines i  and  actual  CNV  (dotted  lines).  Waveforms  are  from  midline  frontal  (Fz).  central  (Czi  and 
parietal  iPzi  electrodes  reterenced  to  linked  ears.  (Adapted  from  Robroaugn  et  al.  1980.) 
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11.4.2.  Probability  and  information:  P300  and  other  late  positive  pe^ks 

Under  appropriate  conditions,  task-relevant  visual,  auditory  or  tactile  stimuli,  or 
the  omission  of  expected  stimuli,  elicit  a  sequence  of  overlapping  positive  peaks 
in  the  300-700  msec  latency  range.  This  is  termed  the  late  positive  complex  (LPC) 
and  includes  the  P3a.  P3b  and  slow  wave  (SW).  The  P3b  peak  (usually  called 
'P300')  was  first  reported  in  1965  (Sutton  et  al.  1965)  and  has  been  the  subject  of 
many  subsequent  studies.  However,  this  robust,  easily  elicited  wave  has  been 
highly  resistant  to  a  precise  definition  of  its  relationship  to  cognition  and  behavior. 
Althougn  P300  is  influenced  by  many  cognitive  factors  in  a  complex  manner,  there 
are  several  well-established  facts  about  its  behavior  (reviewed  in  Donchin  1979; 
Pritchard  1981:  Duncan-Johnson  and  Donchin  1982).  Task-relevant  (‘attended') 
stimuli  elicit  substantial  P300  waves,  and  its  amplitude  is  larger  for  infrequent 
deviant  stimuli,  as  in  the  popular  'oddball'  paradigm,  where  infrequent  target' 
stimuli  must  be  distinguished  from  frequent  'non-targets.'  Its  midline  distribution 
is  maximal  at  Pz  or  Cz  and  is  relatively  unaffected  by  physical  stimulus  properties, 
such  as  modalitv  or  intensity,  provided  that  they  do  not  provide  task-relevant 
information  (Simson  et  al.  1976.  1977a.  b).  The  amplitude  of  P300  is  inversely 
related  to  the  probability  of  occurrence  of  stimuli  which  have  different  'meanings' 
in  the  context  of  the  task  (Tueting  et  al.  1970).  This  effect  of  global  probability 
can  be  modified  by  several  factors.  The  ordering  of  preceding  stimuli  ('local'  or 
'subjective'  probability)  can  override  the  effect  of  global  probability  (K.  Squires 
et  al.  1976).  Similarly.  P300  amplitude  to  feedback  stimuli  is  not  related  to  absolute 
probability,  but  to  the  ‘contingent  probability'  of  a  particular  feedback  following 
a  particular  response  (Campbell  et  al.  1979).  Also,  uncertainty  (‘equivocation  )  in 
difficult  judgment  tasks  weakens  the  effect  of  global  probability  on  P300  (Ruchkin 
and  Sutton  1978). 

P300  is  affected  by  many  other  factors,  and  its  behavior  is  usually  interpreted  in 
terms  of  concepts  borrowed  from  the  field  of  cognitive  psychology  However,  its 
behavior  may  fit  a  particular  cognitive  modei  in  one  study,  but  ;n  a  different 
paradigm  it  is  often  contradictory.  There  seems  to  be  no  simple  relationship  be¬ 
tween  P300  and  a  specific  psychological  factor,  and  because  of  this,  a  parsimonious 
model  of  P300  and  cognition  has  not  yet  been  achieved.  The  difficulty  of  interpre¬ 
ting  P300  in  terms  of  psychological  constructs  is  discussed  in  Chase  et  al.  (1984) 
and  Donchin  (1984).  From  a  neurophysiological  perspective,  there  is  little  reason 
to  expect  that  simple  peak  amplitude  and  latency  measurements  from  a  few  midline 
electrodes  could  characterize  specific  neurocognitive  processes.  Even  so.  substan¬ 
tial  progress  toward  understanding  neurocognitive  processes  has  been  achieved 
through  the  ingenuity  of  the  first  generation  of  ERP  researchers.  We  will  discuss 
several  prominent  lines  of  research  and  the  evolution  of  some  theories  which  used 
to  explain  the  behavior  of  P300  in  order  to  illustrate  the  effect  of  overly  simplified 
ideas  about  cognitive  processing  on  past  ERP  research. 


348 


A.S.  GEVINS  and  B  A.  CITILLO 


11.4.2.1.  P300  and  selective  attention 

Attempts  have  been  made  to  relate  the  behavior  of  P300  to  psychological  models 
of  selective  attention  (reviewed  in  Donald  1983).  Hillyard  and  colleagues  (Hillvard 
et  al.  1973;  Hink  et  al.  1977)  interpreted  the  behavior  of  the  N10Q  and  P300  waves 
in  terms  of  the  ‘stimulus-set/response-set’  model  of  selective  attention  (Broadbent 
1970).  'Stimulus-set  selection'  denotes  early  selection  of  stimuli  in  a  particular 
sensory  'channel'  defined  by  modality,  spatial  location  or  gross  physical  properties, 
and  response-set  selection'  denotes  subsequent  selection  based  on  higher-order 
criteria  related  to  choice  of  appropriate  response.  However,  the  'stimulus-set'  ef¬ 
fect  initially  attributed  to  N100  was  found  to  be  due  to  an  overlapping  negative 
wave,  the  Nd*  or  'processing  negativity'  (Niiatanen  et  al.  1978;  Hansen  and  Hill- 
yard  1980),  and  P300  seems  to  be  relatively  unaffected  by  stimulus-response  com¬ 
parability  (discussed  in  McCarthy  and  Donchin  1983).  and  its  latency  is  not  well 
correlated  with  reaction  time  (see  next  section).  Still,  there  is  ample  evidence  of 
'channel  selection'  in  early  ERP  components  and  higher-order  selection  associated 
with  P300,  but  these  effects  are  not  easily  explained  by  any  of  the  eariy/late  selec¬ 
tion  models  derived  from  psychological  studies  (see  Hillyard  1984;  Posner  et  al. 
1984). 

11.4.2.2.  P300  and  stimulus  evaluation 

A  more  refined  theory  proposes  that  P300  latency  is  related  to  the  time  required 
for  stimulus  evaluation.  If  so,  its  latency  might  be  expected  to  vary  with  the  diffi¬ 
culty  of  stimulus  evaluation  and.  therefore,  with  response  time.  In  relatively  diffi¬ 
cult  judgment  tasks,  its  latency  has  been  reported  to  be  correlated  with  RT.  but 
onlv  moderately,  from  r  =  0.20  to  0.66  (Donchin  et  al.  1978;  McCarthy  and 
Donchin  1983).  In  simpler  tasks  with  short  reaction  times,  P300  often  peaks  after 
response  initiation.  One  proposed  explanation  is  that  the  relationship  of  P300  and 
response  processes  changes  according  to  task  requirements  and  performance 
strategy  (Donchin  1984).  For  example,  when  speed  of  response  is  emphasized,  the 
correlation  of  P300  latency  and  RT  is  reduced,  although  P300  latency  per  se  is 
relatively  unaffected,  but  this  does  not  explain  its  weak  correlation  with  RT  in  the 
same  task,  as  determined  by  single-trial  measurements  (Kutas  et  al.  1977).  Another 
possible  explanation  is  that  P300  is  a  composite  of  several  overlapping  peaks  with 
different  relationships  to  RT.  as  in  the  study  by  Friedman  et  al.  (1978),  in  which 
the  late  positive  wave  consisted  of  4  highly  overlapped  peaks,  only  one  of  which 
was  correlated  with  RT  (see  Fig.  5).  In  fact,  the  correlation  of  the  one  peak  was 
unusually  high  (r  =  0.81-0.90),  suggesting  that  the  weaker  correlations  in  other 
studies  might  be  due  to  failure  to  distinguish  such  overlapping  components. 

Better  support  for  the  stimulus  evaluation  theory  is  provided  by  changes  in  P300 
latency  resulting  from  manipulation  of  factors  affecting  stimulus  evaluation  proces¬ 
ses.  Latency  increases  have  been  produced  by  increasing  the  difficulty  of  stimulus 
discnminability  and  stimulus  categorization.  Discriminabilitv  factors  which  increase 
P300  latency  include  detection  of  small  variations  in  simple  stimuli  (Ritter  et  al. 
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1972;  N.  Squires  et  al.  1977),  stimulus  complexity  (Ritter  et  ai.  1983b),  and  degrad¬ 
ed,  masked  or  •noisy’  stimuli  (McCarthy  and  Donchin  1981;  Ritter  et  al.  1982). 
Factors  related  to  categorization  difficulty  include  difficulty  of  judgment  (Ritter  et 
al.  1983b),  and  number  of  items  in  the  comparison  (memory)  set  (Ford  et  al. 
1979).  However,  these  factors  also  affect  the  N200  waves,  which  always  precede 
response  initiation  (Renault  et  al.  1982)  and  are  better  correlated  with  reaction 
time  (Ritter  et  al.  1979;  Rjtter  et  al.  1983a,  b),  suggesting  that  they  are  more 
closely  associated  with  the  stages  of  stimulus  evaluation  (see  Section  11.5.1.3). 

11.4  2.3.  P300  and  memory :  updating  of  an  ‘internal  model' 

A  possible  explanation  both  for  the  dissociation  of  P3Q0  latency  and  reaction  time 
and  its  relation  to  stimulus  evaluation  is  that  P30Q  is  associated  with  the  updating 
of  an  internal  schema  or  memory  template  subsequent  to  stimulus  evaluation  (Tue- 
ting  1978,  Donchin  1981;  McCarthy  and  Donchin  1983).  This  theory  proposes  that 
the  internal  schema  cannot  be  updated  until  the  significance  of  the  stimulus  has 
been  fully  evaluated,  while  a  response  can  be  initiated  as  soon  as  the  information 
required  for  performance  of  the  current  trial  has  been  evaluated.  Thus  P300  latency 
would  be  correlated  with  reaction  time  indirectly,  and  the  degree  of  correlation 
would  depend  on  the  stage  of  stimulus  evaluation  required  for  response  initiation 
and  context  updating  in  a  particular  task. 

Two  aspects  of  'context  updating'  are  the  amount  of  updating  induced  by  the 
current  stimulus  and  its  effects  on  'expectancy’  which  should  be  evident  in  the 
succeeding  trial.  Since  this  has  not  been  systematically  studied,  we  will  consider 
evidence  from  the  literature  which  bears  on  these  issues.  The  'amount  of  updating' 
aspect  is  supported  by  its  larger  amplitude  for  infrequent  stimuli  ('global  probabi¬ 
lity’  effect)  and  longer  latency  for  disconfirming  feedback  (K.  Squires  et  al.  1973). 
two  instances  where  more  updating  would  be  expected.  Evidence  for  the  'expectan¬ 
cy'  aspect  is  the  influence  of  local  probability,  which  can  modulate  the  effect  of 
global  probability  on  P300  amplitude  (K.  Squires  et  ai.  L976).  The  order  of  prece¬ 
ding  stimuli  seems  to  create  an  expectancy  set.  resulting  in  larger  amplitudes  when 
an  expectation  is  violated,  and  the  weakening  of  the  effect  of  local  probability  at 
long  inter-stimulus  intervals  suggests  the  decay  of  a  memory  trace.  The  expectancy 
theory  would  also  account  for  the  dependence  of  P300  amplitude  for  feedback 
stimuli  on  the  ‘contingent  probability’  with  respect  to  the  preceding  response, 
rather  than  the  absolute  probability  of  a  particular  feedback.  The  interplay  be¬ 
tween  global  and  local  probability  was  demonstrated  by  K.  Squires  et  al.  (1976) 
by  using  a  50/50  probability  of  targets  and  non-targets.  Since  global  probability 
was  equal.  P300  amplitude  in  the  averages  for  targets  and  non-targets  was  the 
same,  but  when  trials  were  averaged  according  to  the  preceding  stimuli.  P300 
amplitude  was  larger  for  targets  which  followed  non-targets  (Fig.  7).  and  even  for 
non-targets  which  followed  targets.  Other  support  comes  from  studies  of  ’equivoca¬ 
tion.'  in  which  P30Q  amplitude  is  directly  related  to  the  degree  of  confidence  with 
which  a  stimulus  can  be  used  in  updating  (reviewed  in  Ruchkin  and  Sutton  1978), 
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and  from  ‘guessing’  paradigms,  in  which  its  amplitude  is  larger  when  the  subject’s 
prediction  is  disconfirmed  (Horst  et  al.  1980). 

Another  line  of  evidence  linking  P300  to  memory  processes  comes  from  studies 
of  short-term  memory  search.  The  most  common  psychological  paradigm  for  study¬ 
ing  short-term  memory  scanning  is  Sternberg's  memory  search  task  (Sternberg 
1969),  in  which  previously  designated  targets  (‘memory  set’)  must  be  identified  in 
a  series  of  random  stimuli  (’test  set’).  This  is  very  similar  to  the  ‘oddball’  paradigm 
used  in  many  P300  studies,  in  which  the  memory  set  (target)  usually  consists  of 
one  item.  Since  the  target  remains  the  same  over  many  trials,  no  updating  of  a 
memory  representation  of  the  target  should  be  required,  yet  P300  amplitudes  are 
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Fig.  7  Effect  of  stimulus  sequence  (  local  probability')  on  P300  amplitude  m  an  auditory  discrimination 
task.  Since  the  ratio  of  targets  and  non-targets  (global  probability)  was  50/50.  P300  amplitude  in 
averaged  ERPs  for  targets  and  non-'  .  ’eis  was  the  same.  However,  when  averaged  according  to  the 
sequence  of  preceding  stimuli.  P300  amplitude  for  both  targets  and  non-targets  increased  linearly  3s  a 
function  of  the  number  of  dissimilar  stimuli  preceding  it.  The  first  letter  at  the  left  of  each  sequence 
indicates  the  letter  of  the  target  in  that  sequence,  i.r..  B'  is  the  target  in  the  first  a  sequences  and  A 
is  the  target  in  the  last  5  sequences.  I  Adapted  from  K.  Squires  et  al.  1976.) 
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larger  for  stimuli  which  match  the  target  (at  target  probabilities  of  50%  or  less). 
Thus  the  amplitude  increase  to  targets  may  be  related  instead  to  the  updating  of 
subjective  expectancy  regarding  the  sequence  of  stimulus  events.  However.  ERP 
studies  using  the  Sternberg  paradigm  suggest  a  relationship  between  P300  and  the 
actual  memory  search  and  comparison  process  (Marsh  1975;  Adams  and  Collins 
1978;  Ford  et  al.  1979).  P300  latency  to  matches  was  found  to  increase  in  a  linear 
manner  with  the  number  of  items  in  the  memory  set  (about  25  msec  per  item).  A 
recent  study  by  Kramer  et  al.  (1983)  produced  several  interesting  results.  The 
slope  of  the  linear  increase  in  P300  latency  with  memory  set  size  for  target  matches 
was  the  same  as  the  slope  for  the  increase  in  reaction  time  (RT),  but  for  non-targets 
(mismatches)  the  RT  slope  was  steeper,  while  the  latency  slope  remained  the 
same.  Tnis  might  indicate  that  the  relationship  between  P300  and  the  processes 
leading  to  response  differs  for  targets  and  non-targets,  although  other  interpreta¬ 
tions  for  such  results  are  possible  (Halliday  et  al.  1984).  P30Q  amplitude  was  larger 
for  targets  than  for  non-targets,  even  when  the  frequency  of  targets  was  80%. 
suggesting  that  the  memory  match  had  a  stronger  effect  than  probability.  In  early 
sessions.  P300  amplitudes  to  memory  matches  were  larger  when  targets  were  in¬ 
frequent.  as  would  be  expected,  but  in  later  sessions  this  probability  effect  was 
absent  for  memory  matches  in  the  easier  'consistent  mapping'  condition.  This  was 
interpreted  as  the  lack  of  need  for  ‘memory  updating'  when  task  performance  is 
'automatized.'  Considering  the  behavior  of  P300  in  these  and  other  studies,  it  is 
clear  that  P300  is  not  simply  related  to  short  term  memory  or  updating  of  an 
internal  model. 

11.4.2.4.  P300.  controlled  processing  and  capacity 

The  psychological  concepts  of  'controlled  processing'  and  'processing  capacity' 
have  been  applied  to  the  behavior  of  P30G.  ‘Controlled  processing'  denotes  a 
limited-capacity  central  process  involving  conscious  effort  (reviewed  in  Moray 
1969;  Kahneman  1973),  and  is  distinguished  from  automatic  processing,  which 
does  not  make  demands  on  ‘processing  capacity'  (Schneider  and  Shiffrin  1977). 
Thus  an  automatic  process  does  not  interfere  with  concurrent  automatic  or  control¬ 
led  processes.  Automatic  processes  may  occur  in  parallel,  while  controlled  proces¬ 
ses  are  considered  serial.  Examples  of  automatic  processes  are  initial  encoding  of 
sensory  information  and  the  execution  of  skilled  motor  actions.  Controlled  proces¬ 
ses  include  memory  rehearsal  and  search,  target  detection,  and  various  types  of 
decision  or  judgment.  Prominent  P300  peaks  are  elicited  when  a  task  requires 
controlled  processing,  and  it  has  been  suggested  that  P300  amplitude  is  related  to 
the  amount  of  controlled  processing  required  in  some  tasks  (Rosier  1983).  although 
contradictory  evidence  is  provided  by  many  reports  of  amplitude  reduction  with 
increasing  task  difficulty  and  amplitude  increase  as  stimulus  probabilities  are  lear¬ 
ned. 

When  two  tasks  performed  simultaneously  both  require  controlled  processing, 
their  interference  is  evident  as  a  decline  in  performance  measures  such  as  reaction 
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time  and  accuracy.  If  P300  is  associated  with  controlled  processing,  it  should  reflect 
such  interference.  Studies  using  dual-task  paradigms  (Isreai  et  al.  1980;  Wickens 
et  al.  1983)  have  demonstrated  that:  P300  amplitude  to  infrequent  targets  in  an 
•oddball’  task  is  reduced  when  a  second  task  is  performed  concurrently,  and  its 
amplitude  to  ‘oddball’  targets  varies  inversely  with  the  difficulty  of  the  second  task 
only  if  it  requires  controlled  processing.  For  example,  increasing  the  difficulty  of 
a  well-learned,  presumably  automatized,  tracking  task  does  not  reduce  P300  ampli¬ 
tude  in  the  concurrent  oddball  task,  but  increased  difficulty  of  a  monitoring  or 
vigilance  task  does.  Further,  P300  amplitudes  in  the  two  tasks  vary  reciprocally  as 
task  difficulty  is  manipulated,  suggesting  that  under  these  conditions  it  indexes  the 
distribution  of  processing  capacity  between  tasks  (Wickens  et  al.  1983). 

11.4.2.5.  The  slow  wave 

Some  tasks  elicit  a  long-duration  slow  wave  (SW).  which  begins  as  early  as  180 
msec  and  peaks  between  500  and  700  msec.  It  has  been  distinguished  from  P300 
by  topography  and  behavior  (reviewed  in  Ruchkin  and  Sutton  1983).  SW  ampli¬ 
tude  is  larger  for  all  task-relevant  stimuli,  whether  or  not  they  are  targets,  and  thus 
can  occur  without  P300  (N.  Squires  et  al.  1975).  Its  amplitude  has  been  reported 
to  be  larger  for  non-targets  in  conditions  where  they  might  require  more  processing 
than  targets  (Friedman  et  al.  1981;  Kramer  et  al.  1983),  and  to  increase  with  task 
difficulty,  while  P300  decreases  (Ruchkin  et  al.  1980a).  The  SW  is  thought  to  be 
associated  with  processes  subsequent  to  stimulus  evaluation  in  the  context  of  the 
current  trial,  possibly  related  to  task  performance  over  many  trials  (Ruchkin  et  al. 
1980b;  Stuss  et  al.  1980). 

An  interesting  finding  is  that  SW  topography  can  change  with  task  requirements. 
Its  midline  distribution  is  typically  negative  at  Fz,  positive  at  Pz,  and  about  zero 
at  Cz.  but  in  the  ‘guess'  condition  of  a  detection  *ask.  its  posterior  positivity  shifts 
forward  (Ruchkin  et  al.  1980b).  Also,  an  experimental  effect  is  sometimes  seen 
only  in  the  frontal  negative  or  parietal  positive  portions.  This  suggests  that  the 
anterior  and  posterior  components  of  the  SW  may  be  generated  by  different  neural 
processes  (Picton  and  Stuss  1980;  Fitzgerald  and  Picton  1981;  Friedman  et  al. 
1981). 


11  _5.  The  variety  of  ERP  components 

This  section  deals  in  some  detail  with  several  topics  in  cognitive  ERP  research  in 
order  to  illustrate  the  basic  information  about  cognitive  processing  which  resear¬ 
chers  must  rely  on  in  applying  new  methods  of  signal  processing  and  analysis. 

11.5.1.  Endogenous  negative  waves 

In  the  past  10  years  the  number  of  reported  ERP  components  has  increased  drama¬ 
tically.  ERP  events  first  thought  to  be  unitary  phenomena  were  later  found  to 
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consist  of  several  waves,  distinguishable  by  topography,  experimental  manipula¬ 
tions  and  analytic  methods.  We  will  discuss  several  negative  waves  which  peak 
between  100  and  400  msec  post  stimulus  (reviewed  in  Naatanen  and  Michie  1979b). 

11.5.1.1.  Processing  negativity 

The  ‘processing  negativity’  rNaatanen  et  al.  1978),  or  *Nd'  (Hansen  and  Hillvard 
1980),  is  a  broad  negative  wave  which  is  larger  in  amplitude  for  stimuli  which  must 
be  selected  on  the  basis  of  simple  physical  properties.  It  is  thought  to  be  a  sign  of 
early  selective  attention,  sometimes  referred  to  as  'channel  selection.'  Its  onset  and 
duration  reflect  the  nature  and  difficulty  of  stimulus  selection,  and  its  topography 
reflects  the  modality  and  side  of  stimulation.  Since  it  begins  as  early  as  60  msec 
and  lasts  for  500  msec,  it  overlaps  the  N100  peak  and  causes  an  apparent  increase 
in  N100  amplitude,  which  initially  led  researchers  to  attribute  the  selective  atten¬ 
tion  effect  to  NI00  (Hillvard  et  al.  1973).  It  was  distinguished  from  N100  by  its 
behavior  in  response  to  experimental  manipulations  and  by  subtracting  the  avera¬ 
ged  ERPs  for  attended  and  unattended  stimuli  (Naatanen  et  al.  197,8.  1981:  see 
Fig.  4.  this  chapter).  The  processing  negativity  itself  may  be  composed  of  several 
components  with  different  topographies  3nd  relationshms  to  task  requirements 
(Hansen  and  Hillvard  1983).  A  sequence  of  negative  peaks  in  this  latency  range 
were  reported  to  respond  differentially  to  selection  of  visual  stimuli  on  the  basis 
of  location,  color,  spatial  frequency  or  orientation  (Harter  and  Guido  lQ80;  Harter 
et  al  1982).  The  relationship  of  these  waves  to  selective  attention  and  sensory 
channels  is  not  well  understood  (discussed  in  Hiilyard  1984.  and  Posner  et  al.  1984). 

11.5.1.2.  Missing  stimulus  negativity 

Tne  missing  stimulus  negativity  ,MSN")  is  pan  of  the  emitted  potential  elicited  by 
occasional  omissions  in  a  regular  sequence  of  stimuli  (Klinke  et  al.  1Q68).  In  this 
case,  the  earlier  'exogenous'  waves  are  absent,  and  the  pureiv  endogenous  MSN 
begins  at  the  time  the  stimulus  was  expected  (see  i  .g.  3).  Its  amplitude  is  largest 
over  the  appropriate  sensor,  cortex  for  expected  visual,  auditory  or  somesthetc 
stimuli  i  Simson  et  al.  1976.  I97"a;  Renault  1983).  and  is  relatively  independent  of 
attention  or  task  requirements  (Naatanen  and  Michie  '979a).  The  MSN  is  similar 
in  topographv  and  behavior  to  the  NZOO  wave  described  below. 

11.5.1.3.  The  N200  peaks 

Inrreacent  stimuli  which  deviate  from  the  majormy  eticii  a  negative  wave  at  about 
ZOO  msec  (Roth  1973;  Squires  et  al.  1975).  Tne  stimuli  need  not  be  task-relevant 
or  attended'  (Snyder  and  Hiilyard  1976).  and  NZOO  amplitude  is  inversely  related 
to  the  frequency  of  deviant  stimuli  (Squires  et  al.  1975,.  Since  NZOO  closely  over¬ 
laps  the  PZOO  peak,  it  is  usually  measured  in  the  difference  waveform  by  subtracting 
the  ERP  to  the  frequent  'standard'  stimuli  from  the  ERP  to  deviant  stimuli. 

The  NZOO  wave  has  been  seen  to  consist  of  two  separate  p<*aks  in  a  number  of 
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studies.  The  N2a  or  ‘mismatch  negativity’  (M.V£N)  is  elicited  by  infrequent  deviant 
stimuli,  task-relevant  or  not.  Its  sensitivity  to  probability  and  magnitude  of  devia¬ 
tion  led  to  the  interpretation  of  N2a  as  a  sign  of  ‘neuronal  mismatch'  in  the  orien¬ 
ting  response  to  deviant  stimuli,  "an  automatic,  basic  sensory  process  ...  irrespec¬ 
tive  of.  perhaps  even  unmodified  by,  the  task  and  subjective  factors’  (Naatanen 
1982).  Its  modality-specific  topography  suggests  involvement  of  activity  in  and 
around  the  sensory  cortex  (Simson  et  ai.  1977a),  and  its  amplitude,  onset  time, 
peak  latency  and  duration  are  affected  by  physical  factors,  such  as  stimulus  discri- 
minability,  complexity,  and  nature  and  magnitude  of  deviation.  The  slightly  later 
N2b  peak  is  not  sensitive  to  these  factors,  but  is  affected  by  certain  task-reiated 
factors  such  as  attention  and  categorization  difficulty  (Ritter  et  al.  1979).  N'2b  has 
a  modality-independent  fronto-central  distribution  similar  to  P3a.  which  appears 
to  always  follow  it  (Squires  et  al.  1975.  1977).  A  similar  pair  of  peaks  have  been 
observed  in  the  missing  stimulus  negativity  (Renault  and  Lesevre  1979),  and  it  has 
been  reported  that  in  certain  tasks  N2b  is  preceded  by  a  negative  peak  (termed 
’NA'),  which  differs  from  N2a  in  that  it  is  not  affected  by  probability  (Ritter  et  al. 
1983a. b). 

Since  N2a  (oi  ‘NA’)  is  affected  by  exogenous  factors,  while  the  behavior  of  N2b 
is  more  endogenous,  and  since  an  experimental  manipulation  which  increases  the 
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Fig  3.  Differential  mamoutauon  of  rwo  N?0Q  peaks.  The  earlier  NA'  wave  is  influenced  bv  the 
exogenous  factor  of  stimulus  discnminaotlity.  while  the  later  'N2'  (Nib)  is  affected  by  the  endogenous 
factor  of  categorization  difficultv  The  bars  connect  onset  and  peak  latencies  of  NA  and  NT.  In  both 
tasks,  increasing  stimulus  discnminaoilitv  by  masking  lengthens  the  onset-to-peak  times  of  NA.  which 
causes  an  identical  delay  in  the  onset  of  N2.  The  increase  in  difficulty  of  judgment  of  stimulus  category 
between  tasks  A  and  B  does  not  affect  the  timings  of  NA.  but  increases  the  peak  latency  and  onset-to- 
peak  :ime  of  N2.  (Adapted  from  Ritter  et  at.  1983a. j 
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latency  of  the  earlier  peak  produces  an  identical  increase  in  the  timing  of  N2b 
(Ritter  et  al.  1979,  1983a;  Renault  1983),  they  are  thought  to  be  associated  with 
successive,  contingent  stages  of  information  processing.  Several  two-stage  models 
have  been  proposed:  pattern  recognition  and  stimulus  categorization  (Ritter  et  al. 
1982.  1983a. b),  two  stages  of  the  orienting  rc.ponse  (Naatanen  and  Gaillard 
1983),  automatic  and  controlled  processing  (Ritter  et  al.  1983b).  and  active  percep¬ 
tual  processing  (‘template  matching')  leading  to  evaluation  of  stimulus  significance 
and  response  selection  (Renault  1983).  All  these  models  propose  that  the  earlier 
wave  is  associated  with  automatic  feature  extraction  processes  which  precede  the 
endogenous  processes  associated  with  N2b.  and  that  their  peak  latency,  onset  and 
duration  reflect  the  timing  of  such  processes.  High  correlations  of  peak  latencies 
and  reaction  times  have  been  reported  (Renault  et  al.  1982;  Naatanen  and  Gaillard 
1983).  and  differential  manipulation  of  the  amplitude  and  timing  of  the  two  waves 
by  changes  in  physical  ('exogenous')  and  task-related  ('endogenous')  factors  sup¬ 
port  this  theory  (McCarthy  and  Donchin  1981;  Ritter  et  al.  1982.  1983a. b;  Fig.  8 
of  this  chapter). 

11.5.2.  Chronometric  models  of  ERPs 

'Chronometncs'  refer  to  methods  for  inferring  the  sequence  and  timing  of  compo¬ 
nent  s:ages  of  cognitive  processing  (reviewed  in  Chase  1984).  This  approach  was 
pioneered  by  Donders  in  the  mid-1800's  and  has  been  most  elegantly  applied  in 
the  work  of  Posner  (1978).  Chronometric  analysis  is  well  suited  to  ERP  studies, 
since  the  timing  of  onset,  peak  latency  and  duration  of  the  waves  associated  with 
various  cognitive  factors  can  be  used  to  infer  the  temporal  relationships  of  proces¬ 
sing  stages.  The  short  and  medium  latency  peaks  in  the  First  80  msec  following  a 
stimulus  are  highly  exogenous:  their  latency  and  topography  depend  on  the  moda¬ 
lity.  intensity,  and  other  physical  properties  of  the  stimulus.  The  PlOO  and  N100 
peaks  are  also  considered  exogenous,  generated  primarily  m  sensory  cortex,  al¬ 
though  it  is  likely  that  other  conical  areas  are  also  involved  (Goff  et  al.  1978: 
Wood  et  al.  1 QS0;  Allison  1984).  They  are  thought  to  reflect  later  stages  of  feature 
encoding.  The  P200  peak  which  immediately  follows  NTOO  is  usually  considered 
“xoaenous.  although  us  topography  is  less  dependent  on  stimulus  modality  and 
not  largest  over  sensory  conex.  If  the  task  requires  selection  of  stimuli  on  the  basis 
of  simple  physical  propenies.  the  long-duration  ‘processing  negativity,-  thought  to 
be  a  sign  of  early  selective  attention,  may  occur  in  this  latency  range  (Naatanen 
et  al.  1978). 

If  the  stimu'"s  differ*  from  the  majority  of  stimuli,  the  N2-P3a  peak  compiex 
occurs  in  the  _u<)-300  msec  range  (N.  Squires  et  al.  1975).  Task  relevance  (‘atten¬ 
tion)  is  not  required,  and  their  amplitudes  vary  inversely  with  probability.  NTOO 
consists  of  two  successive  peaks.  The  earlier  peak,  termed  N2a.  mismatch  negati¬ 
vity  and  NA  by  various  authors,  is  affected  by  exogenous  factors  such  as  stimulus 
modality,  complexity,  discnminabilitv.  and  nature  and  magnitude  of  deviation, 
whereas  NZb  is  sensitive  to  higher-order  task  factors  such  as  difficulty  of  stimulus 
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categorization.  Also,  since  the  timing  of  N2b  is  affeaed  by  changes  in  latency  of 
the  earlier  wave,  and  N2b  latency  is  highly  correlated  with  reaction  time.  N2a  (or 
NA)  and  N2b  are  thought  to  reflect  successive,  contingent  stages  of  automatic 
feature  recognition  (template  matching)  and  subsequent  task-related  evaluation 
(see  Section  11.5.1.3). 

If  the  stimulus  is  also  task  relevant.  P3a  will  be  followed  by  the  P3b  (P300)  peak, 
usually  largest  over  parietal  (‘association’)  areas.  The  behavior  of  P3b  is  affected 
in  a  complex  manner  by  numerous  cognitive  factors  of  a  higher  order  than  the 
preceding  waves  (see  Section  11.4.2).  P3b  appears  to  be  involved  in  almost  all 
types  of  higher  cognitive  functions.  Some  tasks  elicit  later  peaks  such  as  the  N400. 
thought  to  be  associated  with  semantic  processing  (see  Section  11.5.3.1).  and  the 
long-duration  slow  wave,  peaking  between  500  and  700  msec,  which  seems  associat¬ 
ed  with  processes  subsequent  to  stimulus  evaluation  in  the  current  trial,  possibly 
related  to  task  performance  over  many  trials. 

A  typical  visual  discrimination  task  might  consist  of  identifying  designated  target 
stimuli  which  occur  infrequently  in  a  sequence  of  standard'  stimuli,  and  making  a 
finger  movement  in  response  to  targets.  The  averaged  ERP  for  target  stimuli  might 
contain  a  sequence  of  peaks  like  those  at  the  top  of  Fig.  9:  P90.  N130.  P190.  N225 
(N2a),  N250  (N2b).  P290  (P3a),  P375  (P3b).  a  slow  wave  (SW)  peaking  at  575 
msec,  and  negative  movement-related  potentials  (RP)  peaking  at  500  msec.  The 
chronology  of  onset,  peak  latency  and  duration  of  the  component  waves  in  this 
hypothetical  waveform,  as  might  be  derived  by  PCA.  subtraction,  topographic,  or 
other  methods,  is  illustrated  in  the  lower  pan  of  Fig.  9.  These  component  waves, 
which  are  associated  with  various  aspects  of  the  stimulus  and  task  processing,  can 
be  seen  to  wax  and  wane  in  a  highly  overlapping  manner.  Since  the  onset  of  a 
particular  wave  often  precedes  the  peak  of  the  previous  wave,  it  appears  that  the 
neural  process  indexed  by  the  later  wave  begins  before  the  culmination  of  the 
preceding  stage,  which  agrees  with  evidence  from  chronometnc  behavioral  studies 
that  cognitive  processing  involves  multiple  parallel  and  sequential  stages. 

Most  models  applied  to  ERPs  to  describe  the  flow  of  information  between  stages 
are  derived  from  psychological  studies  (discussed  in  Chase  et  al.  1984).  The  as¬ 
sumption  of  strictly  sequential  processing  stages,  as  in  Sternberg's  ‘additive  factor 
method'  (Sternberg  1969),  seems  unsuited  to  most  ERP  phenomena.  Several  mo¬ 
dels  combining  parallel  and  sequential  views  have  been  considered,  including 
McClelland’s  ‘cascade  model'  (McClelland  1979).  the  'asynchronous  discrete  stage 
model’  of  Miller  (1982).  and  others  based  on  similar  principles,  such  as  ‘contin¬ 
uously  available  output’  (Norman  and  Bobrow  1976)  and  'continuous  flow’  (Enk- 
sen  and  Schultz  1979).  The  approach  of  Posner,  though  not  expressed  as  a  general 
model,  is  perhaps  the  most  promising  for  chronometnc  ERP  analysis  (Posner 
1973). 

Considenng  the  hypothetical  ‘components'  of  the  ERP  waveform  for  target 
stimuli  of  our  visual  discrimination  task  (Fig.  9)  in  terms  of  information  flow  and 
processing  stages,  we  might  speculate  that  the  encoding  of  stimulus  features  begins 
with  the  mid-latency  exogenous  waves  between  40  and  80  msec  and  continues 
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through  P90,  N130  and  P190.  The  feature  codes  necessary  for  identification  of 
target  stimuli  must  be  available  by  the  N2a  peak  at  225  msec,  since  N2a  (or  ‘NA’) 
is  affected  by  the  nature  and  degree  of  deviation  from  the  ‘standard’  (i.e.,  the 
mismatch  between  feature  code  and  internal  ‘neural  template’  of  the  frequent 
‘standard’  stimuli).  This  idea  is  supported  by  the  increase  in  latency  and  duration 
of  NA  caused  by  degraded  or  complex  stimuli,  which  may  require  more  processing 
to  generate  a  utilizable  feature  code.  Such  an  increase  in  timing  delays  the  succeed¬ 
ing  N2b  by  the  same  amount,  indicating  that  processes  associated  with  N2b  are 
contingent  on  those  associated  with  the  earlier  wave.  N2b  itself  is  sensitive  to  more 
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Fig.  9  Top:  model  avenged  ERP  waveform  for  infrequent  target  stimuli  m  a  hypothetical  visual 
discrimination  task,  showing  mid-latency.  Pt.  Nl.  P2.  NLa.  N2b,  P3a.  P3b  and  slow  wave  ($W)  peaks, 
the  processing  negativity.'  and  the  movement-related  readiness  potential  (RP).  Bottom:  hypothetical 
components'  of  the  waveform,  as  might  be  interred  by  subtraction.  PCA  and  topographic  methods, 
showing  their  onset  times,  peas  latencies  and  durations.  Note  the  high  degree  of  overlap  of  successive 
components.  (See  text  for  further  discussion. ) 
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'endogenous'  task-related  factors  which  do  not  affect  N2a.  The  topographies  of 
N2b  and  the  similarly  behaving  P3a  which  follows  it  are  not  affected  by  stimulus 
modality,  suggesting  that  they  are  associated  with  processes  involving  a  modality- 
independent  or  ‘common  code.'  The  increase  in  N2b  latency  caused  by  categoriza¬ 
tion  difficulty  and  its  high  correlation  with  reaction  time  suggests  that  it  is  as¬ 
sociated  with  evaluation  of  stimulus  information  required  for  response  selection. 
The  P3b  (‘P30G')  peaks  at  375  msec  and  is  elicited  only  by  task-relevant  stimuli. 
Its  latency  is  sometimes  (but  not  always)  moderately  correlated  with  N2b  latency 
and  reaction  time.  Its  amplitude  is  larger  for  stimuli  which  match  the  memorized 
target,  but  also  becomes  larger  as  the  frequency  of  target  stimuli  is  reduced.  Con¬ 
sidered  along  with  its  latency  behavior  (see  Sections  11.4.2.2  and  3),  P3b  appears 
to  be  related  both  to  stimulus  evaluation  in  the  current  tnal  and  also  to  the  updat¬ 
ing  of  an  internal  schema  or  expectancy  set  subsequent  to  stimulus  evaluation. 
That  is.  it  seems  to  be  contingent  on  ‘common  codes'  associated  with  stimulus 
evaluation  (possibly  indexed  by  N2b).  but  utilizes  those  codes  for  updating  an 
internal  schema  which  is  involved  in  task  performance  over  a  series  of  trials.  The 
slow  wave  also  seems  related  to  processes  subsequent  to  stimulus  evaluation:  it 
may  also  be  involved  in  ongoing  task  performance,  perhaps  more  than  P3b.  since 
it  is  strong  for  non-targets  as  well  as  targets. 

11.5.3.  Language  and  lateralization 
11.5.3.1.  S4O0 

Recently,  a  negative  wave  peaking  at  about  400  msec  was  reported  to  be  related 
to  semantic  incongruity'  in  the  reading  of  sentences  (Kutas  and  Hillvard  !980a. 
b).  This  N400'  wave  was  elicited  by  a  word  which  was  semantically  inappropriate 
in  the  context  of  a  sentence.  A  semantically  appropriate  word  presented  in  a  larger 
typeface  elicited  a  P300  wave  without  N400.  suggesting  that  deviation  from  seman¬ 
tic  context  was  required  for  elicitation  of  N400.  In  support  of  this  interpretation. 
N400  was  found  to  be  relatively  insensitive  to  the  probability  of  semantically  inap¬ 
propriate  words,  and  also,  violations  of  grammar  did  not  elicit  an  N'400-like  wave 
(Kutas  and  Hillvard  1982).  Later  studies  (Kutas  et  al.  1984)  indicated  that  N400 
amplitude  was  related  to  semantic  expectancy .  rather  than  semantic  mappropnate- 
ness  per  se:  appropriate  words  with  a  low  degree  of  semantic  expectancy  (Cloze 
probability!  also  elicited  N400.  although  smaller  in  amplitude  than  for  inappropri¬ 
ate  words.  Similar  waves  have  been  reported  fer  semantic  mismatch  in  word  pain, 
for  words  outside  an  expected  category,  and  for  semantic  judgments  of  picture 
stimuli  (reviewed  in  Kutas  et  al.  1984).  Such  findings  led  to  the  hypothesis  that 
N'400  reflects  priming  or  facilitation  in  semantic  processing. 

Semantic  expectancy  may  not  be  the  only  factor  which  elicits  N400.  Similar 
negative  waves  have  been  produced  by  naming  single  words  and  pictures,  where 
there  is  no  semantic  context,  and  by  non-semantic  tasks  such  as  mental  rotation 
and  comparison  of  geometrical  figures  (Stuss  et  al.  1983).  It  has  been  suggested 
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that  N400  reflects  the  utilization  of  short-term  working  memory  in  maintaining  a 
task-related  context,  semantic  or  otherwise,  which  would  certainly  be  involved  in 
the  processing  of  long  sentences. 

11.5.3.2.  Lateralization  studies 

The  right-sided  lateralization  of  N400  (Kutas  and  Hillvard  1982)  is  difficult  to 
interpret  in  view  of  the  abundant  neuropsychological  evidence  for  the  involvement 
of  certain  left-hemisphere  conical  areas  in  linguistic  functions.  The  well  substantia¬ 
ted  role  of  the  ‘language'  areas  of  posterior  frontal  and  temporal  cortices  of  the 
dominant  hemisphere  (usually  the  left  hemisphere  in  right-handed  persons')  makes 
language  a  good  choice  for  studying  the  localization  of  neurocognitive  processes. 
Many  attempts  have  been  made  to  measure  language-associated  lateralization  and 
localization  in  the  EEG  and  ERP.  but  the  findings  have  been  inconsistent  (re¬ 
viewed  in  Beaumont  1983:  Gevins  1983;  Molfese  1983;  Rugg  1983;  Butler  and  Giass 
1986).  Topics  which  have  been  addressed  include  acoustic  and  phonetic  processing 
(Wood  1975:  Molfese  1980).  the  effect  of  semantic  context  on  homonyms  (Brown 
et  al.  1979).  connotative  (affective)  value  (Chapman  1977;  Chapman  et  al.  1980). 
and  lexical  and  semantic  functions  (Buchsbaum  and  Fedio  1970:  Brown  et  al.  1976: 
Chapman  et  al.  1978;  Molfese  1979).  The  senes  of  studies  by  Kutas  and  colleagues 
are  particularly  interesting  and  include,  in  addition  to  the  N400  studies  mentioned 
above,  studies  of  natural  sentence  processing,  grammar,  and  differences  between 
processing  of  open-class  (content)  and  closed-class  (function)  words  (Kutas  and 
Hillvard  1980a. b.c.  1982:  Kutas  et  al.  1984). 

A  number  jf  studies  have  reported  language-associated  hemispheric  lateraliza¬ 
tion  of  one  or  more  ERP  peaks,  often  with  larger  amplitudes  over  left  hemisphere 
sites  of  right-handed  people.  Unfortunately,  many  of  the  studies  are  of  question¬ 
able  validity  due  to  methodological  flaws  (reviewed  in  Friedman  et  al.  1975:  Don- 
chm  et  al.  197“;  Gevins  et  al.  1979c.  1980:  Beaumont  1983;  Gevms  1983).  Common 
shortcomings  include  failure  to  properly  isolate  the  intended  linguistic  function 
due  to  poor  experimental  design:  inadequate  control  of  spurious  factors  such  as 
stimulus  properties,  eye  movement,  response  movements.  sub-voc3lization  and 
task  difficulty;  and  inappropriate  location  of  the  reference  electrode.  Also, 
anatomical  differences  between  hemispheres  in  the  temporo-panetal  region  must 
be  taken  into  account  in  evaluating  interhemisphenc  differences  between  elec¬ 
trodes  situated  over  the  posterior  language  center'  (Wernicke  s  area).  An  illustra¬ 
tion  of  the  importance  of  experimental  control  is  provided  by  two  studies  of  Gevins 
and  colleagues  (Gevins  et  al.  1979a. b.c).  The  differences  in  lateralization  of  EEG 
spectral  power  between  verbal  and  non-verbal  (spatial)  tasks  found  in  the  first 
study  were  not  observed  in  a  subsequent  study  :n  which  task  difficulty,  stimulus 
and  movement-related  factors  were  more  highly  controlled. 

Other  reasons  for  the  failure  to  find  consistent  patterns  of  lateralization  and 
localization  related  to  linguistic  and  non-linguistic  functions  include  the  use  of 
paradigms  based  on  simplistic  neurocognitive  models,  and  inappropriate  or  in- 


360 


A.S.  GEVINS  and  B  A.  CLTILLO 


adequate  brain  potentiai  measures  and  anaJyses.  It  is  unrealistic  to  assume  that 
just  one  or  rwo  cortical  areas  are  involved  in  any  specific  cognitive  function  (Luria 
1977).  In  the  case  of  language,  regional  cerebral  blood  flow  (rCBF)  studies  have 
shown  that  during  language  tasks  the  pattern  of  cortical  activation  in  the  non-do- 
minani  hemisphere  is  almost  a  mirror  image  of  the  activation  of  the  ‘language 
areas’  of  the  dominant  hemisphere  (Lassen  et  a 1.  1978).  Thus  the  brain  potentials 
associated  with  this  bilateral  activation  might  not  exhibit  gross  lateralization  of 
amplitudes,  and  the  measurement  of  any  subtle  hemispheric  differences  might 
require  more  detailed  spatial  sampling  and  more  sophisticated  methods  of  signal 
processing  and  analysis.  For  example,  in  a  study  companng  split-second  numeric 
and  spatial  judgment  tasks.  Gevins  and  colleagues  (1981)  found  no  significant 
lateralizations  of  peak  amplitudes  in  the  averaged  ERP.  However,  when  a  single- 
trial  measures  of  waveshape  similarity  between  electrodes  (cross-correlation)  in 
brief  analysis  intervals  spanning  the  major  ERP  peaks  were  analyzed  by  mathemat¬ 
ical  pattern  recognition  techniques,  they  found  complex  patterns  of  evoked  corre¬ 
lations  which  differed  between  tasks.  The  lateralization  of  correlations  were  mixed, 
although  in  two  intervals  they  were  predominately  right  or  left  sided. 


11.6.  Advances  in  recording  and  analysis 

A  major  research  challenge  is  the  development  of  better  methods  of  resolving  the 
complex  signals  of  cognition,  which  overlap  both  temporally  and  spatially  at  the 
scalp.  Progress  is  being  made  in  several  areas,  including  ERP  estimation,  post-pro¬ 
cessing.  spatial  sampling  and  analysis,  source  localization,  and  the  modeling  of 
neurocognitjve  processes. 

11.6.1.  Improved  ERP  estimation 

Although  the  averaging  technique  is  useful,  its  effectiveness  is  limited  by  tnal-to- 
irial  variability  in  the  timing,  shape  and  strength  of  event-related  signals  (see 
Chapter  5  of  Vol.  1).  Considerable  effort  has  been  directed  toward  improving 
methods  of  event-related  signal  measurement  and  ERP  estimation,  but  although 
some  techniques  are  applicable  in  particular  instances,  no  single  method  which  can 
replace  averaging  has  been  developed  (see  review  in  Gevins  1984).  Methods  for 
reducing  the  effect  of  ‘atypical’  trials  in  averages,  such  as  down-weighting  (Gasser 
et  al.  1983)  or  elimination  of  outliers  (Gevins  et  al.  1986;  see  Chapter  5  in  Vol.  1 ) 
are  simple  and  fairly  effective.  The  use  of  Wiener  filtering  (also  called  'minimum 
mean  square  error.'  or  MMSE)  to  improve  the  event-related  signal  in  the  average 
ERP  has  been  considered  (see  Chapter  5  in  Vol.  1.  Chapter  10  of  this  Volume,  and 
reviews  in  De  Weerd  1981  and  McGillem  et  al.  1981).  but  their  usefulness  in  study¬ 
ing  neurocognnive  signals  has  not  been  demonstrated.  Latency  correction  proce¬ 
dures  have  been  used  to  compensate  for  tnal-to-tnal  variations  in  the  timing  of 
event-related  signals  by  shifting  the  temporal  position  of  individual  trials  before 
averaging  (Woody  1967).  However,  for  these  procedures  to  be  effective,  the  event- 
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related  signal  must  be  relatively  large  with  respect  to  background  EEG,  and  filter¬ 
ing  of  rhythmic  background  EEG  is  necessary  (see  Section  11.2.1.2).  A  possible 
imorovement  might  be  the  application  of  latency  correction  only  to  trials  with 
distinct  event-related  signals.  Since  the  timing  and  shape  (i.e.,  frequency)  of  in¬ 
dividual  peaks  in  the  ERP  waveform  can  vary  independently,  it  is  clear  that  the 
ERP  is  not  a  stationary  process,  and  recently  developed  time-varying  filters  (see 
Fig.  1.  Chapter  10)  may  provide  more  accurate  measurement  of  neurocognitive 
signals  (De  Weerd  1981;  Yu  and  VlcGillem  1983).  Significant  advances  in  ERP 
signal  estimation  should  occur  when  analytic  models  with  greater  neurophysi¬ 
ological  fidelity  are  developed  (see  11.6.5  below). 

11.6.2.  Improved  post-processing 

Another  area  where  improvement  is  needed  is  feature  extraction,  which  is  the 
determination  of  those  signal  properties  which  best  characterize  the  cognitive  fac¬ 
tors  defined  by  the  experimental  conditions.  Heuristic  feature  extraction,  the  most 
common  method,  consists  of  manually  or  automatically  measuring  the  latency  and 
amplitude  of  an  ERP  peak  of  interest.  However,  overlapping  components  can 
render  such  measures  ambiguous,  and  simple  heuristic  methods  may  extract  only 
a  small  portion  of  the  information  in  the  ERP.  PCA  is  commonly  used  as  an 
'objective'  method  of  ERP  feature  extraction  (Harman  1976;  see  Chapters  3.  16 
and  17  in  Vol.  1)  and  has  been  used  in  many  studies  (Donchin  1966.  1979:  Thatcher 
and  John  1977;  Donchin  et  al.  1978).  However,  a  number  of  somewhat  arbitrary 
decisions  must  be  made  in  applying  PCA.  including  the  segment  of  the  time  senes 
to  transform,  the  type  of  filters  to  be  used,  the  number  of  channels,  expenmental 
conditions  and  persons  to  include,  and  the  type  of  rotation  to  apply  to  the  extracted 
components.  These  decisions  arc  crucial,  since  mixed  sources  of  variance  in  the 
data  can  cause  confusion  in  interpreting  the  meaning  of  the  extracted  components 
(Van  Rotterdam  1970;  Rosier  and  Manzev  1981;  Wood  and  McCarthy  1984).  As 
a  means  of  statistical  feature  extraction,  the  major  drawback  of  PCA  is  that  the 
small  amount  of  residual  variance  unaccounted  for  by  the  major  components  might 
be  crucial  in  characterizing  and  distinguishing  the  cognitive  functions  being  studied 
(see  Chapter  17  in  Vol.  1).  Also,  linear  combinations  of  variables  may  not  be  as 
effective  as  other  combinations  formed  by  procedures  which  do  not  assume  linea¬ 
rity  (Gevins  1980.  1984). 

As  an  alternative  to  the  use  of  PCA  for  feature  extraction,  mathematical  pattern 
recognition  (PR)  procedures  have  been  successfully  used  to  choose  ERP  features 
and  feature  combinations  which  are  good  or  even  optimal  for  hypothesis  testing 
(see  Chapter  17  in  Vol.  1).  One  such  algorithm,  known  as  stepwise  Jiscnminant 
analysis  (SWDA)  (Sampson  1967),  is  widely  available  and  has  been  extensively 
used  for  feature  selection  and  hypothesis  nesting  (Donchin  1966:  D  O.  Walter  et 
al.  1967;  reviews  in  Gevins  1980.  1984;  McGillem  et  al.  1981).  SWDA  first  chooses 
the  vartaole  with  the  greatest  ratio  of  between-condition  to  wuhm-condition  va¬ 
riance  and  the  selects  succeeding  variables  by  the  same  criterion  after  removing 
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the  variance  accounted  for  by  variables  already  chosen.  Despite  its  popularity. 
SWDA  and  other  methods  which  choose  variables  one  at  a  time  have  a  major 
limitation:  the  set  of  best  variables  chosen  one  at  a  time  is  unlikely  to  be  as  good 
as  a  set  of  variables  chosen  in  combination  (Cover  1974;  Cover  and  Van  Campen- 
hout  1976).  The  only  way  to  determine  the  best  combination  of  variables  is  to  try 
all  possible  combinations,  but  this  is  computationally  impractical  in  most  cases. 
Algorithms  which  efficiently  compute  all  possible  linear  subsets  of  variables  have 
been  developed  (Fumival  and  Wilson  1974),  but  the  problem  remains  when  there 
arc  many  variables  or  when  non-linear  combinations  are  desired.  In  such  cases, 
carefully  considered  temporal  or  anatomic  constraints  must  be  used  to  limit  the 
search  for  variable  subsets.  Anatomical  constraints  can  be  applied  by  forming 
neurophystologicallv  relevant  subsets  of  electrodes,  and  temporal  constraints  can 
be  applied  by  limiting  the  search  to  brief  time  intervals  determined  from  the  major 
peaks  in  the  averaged  ERP  (Gevins  et  al.  1979a.  1981,  1985). 

Regardless  of  the  feature  extraction  procedure  used,  the  underlying  problem  is 
that  statistically  independent  features  are  not  necessarily  neurophysiologically  inde¬ 
pendent.  For  example,  if  an  experiment  causes  the  event-related  activity  of  several 
brain  systems  to  covary  together,  only  one  statistically  independent  feature  may  be 
required  to  characterize  the  activity  of  the  several  systems.  There  is  at  present  no 
general  solution  to  this  problem  other  than  to  design  experimental  conditions 
which  may  highlight  differences  between  neural  systems,  and  to  organize  the  data 
for  analysis  using  a  priori  neuroanatomical  and  neurophysiological  knowledge 
(Gevins  et  al.  1979a.  1981.  1985). 

11.6.3.  Spatial  analysis 
11.6.3.1.  Need  for  more  elertrodes 

The  resolution  of  neurophysiologically  distinct  'components'  is  perhaps  the  major 
problem  in  ERP  research.  Much  effort  is  devoted  to  devising  variations  of  a  few 
basic  paradigms,  but  little  to  increasing  the  dimensionality  of  parametrization  of 
event-related  brain  signals.  Small  changes  in  peak  amplitude  or  latency  at  a  few 
midline  electrodes  are  assumed  to  be  adequate  to  characterize  the  mass  neural 
activity  of  a  large  number  of  psychological  functions.  For  example,  since  the  P300 
wave  is  affected  by  many  factors,  how  can  it  be  determined  whether  the  P300 
elicited  in  one  paradigm  is  the  same  P300  observed  in  other  paradigms,  or  a 
different  one  associated  with  different  neurai  processes?  The  P30Q  elicited  by  ‘ no- 
go'  stimuli  in  a  go/no-go  task  is  more  antenor  in  distribution  than  the  'typical'  P3 
(Simson  et  al.  1977b).  and  the  frontal  or  ’novelty’  P300  elicited  by  unexpected 
stimuli  is  even  more  anterior  (Courchesne  et  al.  1975).  Do  such  variations  reflect 
changes  in  the  activity  of  the  same  neural  sources,  or  the  involvement  of  different 
sources?  Information  from  a  larger  number  of  electrodes,  especially  lateral  ones, 
could  shed  some  light  on  this  issue,  but  very  few  studies  use  more  than  the  usual 
3  or  4  midline  sites.  For  instance,  although  the  midline  distribution  of  P300  is 
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unaffected  by  stimulus  modality,  modality  effects  in  its  lateral  topography  have 
been  reported  (Snyder  et  al.  1980).  Studies  using  up  to  50  electrodes  indicate  that 
a  high  degree  of  spatial  sampling  is  useful  in  distinguishing  ERP  components  (Leh¬ 
mann  1981,  1984;  Gevins  et  al.  1984;  Lehmann  and  Skrandies  1984;  see  also  Chap¬ 
ter  12  in  Vol.  1). 

The  spatial  patterning  of  scalp-recorded  brain  signals  is  complex,  reflecting  the 
activity  of  many  anatomically  and  functionally  differentiated  neural  systems.  Re¬ 
cordings  of  many  EEG  channels  simultaneously  are  becoming  more  common  due 
to  advances  in  technology  (Lehmann  et  al.  1969;  Lehmann  1971.  1984;  Kavanagh 
1972;  Kavanagh  et  al.  1978;  Darcev  1979;  Darcey  et  al.  1980;  Ary  et  al.  1981; 
Gevins  et  al.  1984).  Basic  methodological  considerations,  such  as  the  separation 
between  electrodes  required  to  avoid  spatial  aliasing  must  be  considered  (see  Vol. 

I.  Chapters  3  and  12).  With  60  electrodes  evenly  distributed  over  the  head,  the 
average  inter-electrode  distance  is  about  3.25  cm;  with  120  electrodes,  the  distance 
is  about  2.3  cm.  Preliminary  analyses  of  the  spatial  spectrum  of  endogenous  ERPs 
elicited  by  unoattemed  visual  stimuli  indicate  that  a  spacing  of  about  2  cm  is 
necessary  to  avoid  appreciable  spatial  aliasing  (Doyle  and  Gevins  submitted). 

II. 6.3.2.  Removing  the  effect  of  the  reference  electrode 

The  location  of  the  reference  electrode  greatly  affects  the  topography  of  brain 
potentials.  A  P3Q0  peak  recorded  with  mastoid  or  chest  reference  has  a  maximum 
positive  amplitude  at  Cz  or  Pz.  but  is  more  posterior  with  a  nose  reference,  and 
becomes  a  negative  peak  with  an  occipital  maximum  when  a  common  average 
reference  is  used. 

The  Laplacian  derivation  (LDj  promises  to  be  useful  in  removing  the  influence 
of  the  reference  location,  and  also  in  reducing  the  blurring  effect  of  volume  conduc¬ 
tion  (Hjorth  1975;  Nunez  1981).  This  technique  uses  the  potentials  recorded  from 
a  local  group  of  electrodes  to  compute  the  curvature  of  the  electrical  field  at  the 
center  electrode,  yielding  a  measure  of  electrical  current  perpendicular  to  the  scalp 
at  that  site.  LD  measures  are  independent  of  the  location  of  the  reference  elec¬ 
trode.  and  are  less  sensitive  to  current  sources  outside  the  boundary  of  the  local 
electrode  group.  Tne  ability  of  the  LD  to  sharpen  spatial  detail  and  enhance  weak 
signals  in  the  EEG  has  been  demonstrated  (Spehr  1976;  Murray  and  Cobb  1980; 
Wallin  and  Stalberg  1980).  It  has  only  recently  been  applied  to  ERPs.  and  seems 
effective  in  sharpening  the  spatial  pattern  of  exogenous  components  for  visual 
tumuli  (Thickbroom  et  cl.  1984),  and  auditory  stimuli  and  movement-related  sig¬ 
nals  (Gevins  et  al.  1984).  The  LD  distributions  of  endogenous  components  are  not 
as  localized,  but  the  overlapping  late  positive  peaks  are  more  distinguishable  by 
their  LD  topography  (Gevins  et  al.  1984).  Computation  of  the  LD  amounts  to 
little  more  than  adding  the  bipolar  EEG  channels  surrounding  a  particular  elec¬ 
trode.  each  weighted  by  the  inter-electrode  distance,  but  the  LD  works  best  when 
manv  channels  are  recorded  and  the  LD  cannot  be  accurately  determined  for 
channels  on  the  penpnery  of  the  montage.  Spatial  deconvolution  procedures  are 
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also  reference  independent  and  provide  even  more  reduction  of  volume  conduction 
effects  than  the  LD  (Doyie  and  Gevins  submitted). 

11.6.3.3.  Analyzing  spatial  information 

The  first  step  in  spatial  analysis  is  computation  of  the  topographic  pattern  of  ERP 
voltages  (Lehmann  et  al.  1969;  Lehmann  1971;  Kavanagh  1972).  Topographic 
maps  consist  of  isopotential  contour  lines  representing  the  peaks  and  valleys  of 
potential  or  current,  much  like  topographic  geophysical  maps.  Since  the  location 
of  the  reference  electrode  has  a  strong  effect  on  scalp  topography  (Lehmann  1984. 
Lehmann  and  Skrandies  1984;  see  Chapter  12  in  Vol.  1),  voltages  should  be  con¬ 
vened  to  reference-independent  measures  by  Laplactan  denvation  or  spatial  de- 
convolution.  Companson  of  topographic  patterns  across  persons  is  problematic; 
scalp  electrodes  positioned  by  reference  to  skull  landmarks  are  not  precisely  coor¬ 
dinated  with  underlying  conical  areas,  the  variation  between  a  scalp  site  and  a 
designated  cortical  area  being  about  1  cm.  In  addition,  functional  neuroanatomy 
differs  among  people,  so  that  activity  at  a  particular  electrode  may  not  be  function¬ 
ally  equivalent  across  persons.  One  solution  is  the  use  of  magnetic  resonance 
imaging  (MRI)  to  determine  the  relative  position  of  scalp  electrodes  and  cortical 
areas  (Doyle  and  Gevins  submitted). 

While  valuable  insights  can  be  gained  by  examining  topographic  maps,  statistical 
methods  are  needed  to  quantify  spatial  information.  One  approach  is  to  reduce 
the  information  for  statistical  analysis  to  a  few  features,  such  as  the  location, 
orientation  and  magnitude  of  an  ‘equivalent  current  dipole'  (Kavanagh  1972;  Kava¬ 
nagh  et  al.  1978;  Darcey  1979;  Darcev  et  al.  1980)  or  the  locations  of  maximum 
positive  and  negative  potential  values  (Lehmann  1971;  Lehmann  and  Skrandies 

1980.  1984).  Multivariate  statistical  methods  can  also  be  used  to  evaluate  spatial 
patterns  by  reference  to  normative  values  from  control  populations  (John  et  a). 
1977a.  b.  1983;  Duffy  et  al.  1981 ).  or  by  making  comparisons  between  experimen¬ 
tal  conditions  for  each  electrode  or  group  of  electrodes  (Gevins  et  al.  1979a. b.c. 

1981.  1983.  1985).  These  and  other  aspects  of  spatial  analysis  are  discussed  in 
detail  in  Vol.  1.  Chapters  3.  12.  17  and  18  (see  also  Section  10.5  of  Chapter  10). 

11.6.3.4.  Measurement  of  inter-channel  relationships 

Studies  of  the  inter-relationship  of  event-related  activity  between  different  scalp 
locations  aim  to  find  some  indication  of  ‘functional  coordination'  or  ‘interdepen¬ 
dency’  berween  neural  areas.  Such  information  is  potentially  useful  for  inferring 
the  presence  of  several  simultaneously  active  sources.  There  are  two  wavs  of  doing 
this;  the  characteristics  of  the  ERP  waveforms  may  be  quantified  for  individual 
channels  and  the  resulting  measures  compared  between  channels,  or  the  relation¬ 
ship  between  channels  can  be  directly  quantified  by  various  measures  of  similarity 
or  interdependence  of  their  activity,  with  particular  emphasis  on  the  time  lag 
between  channels.  Some  of  the  measures  of  inter-channel  interdependence  inclu¬ 
de:  spectral  coherence  (D  O.  Walter  1963:  see  Chapter  4,  Vol.  1);  polycoherence 
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(Saltzberg  1985;  ses  Chapter  4,  Vol.l);  partial  coherence  (Gersch  and  Goddard 
1970;  see  Chapter  10  in  Voi.  1);  information-theoretic  measures  (Callaway  and 
Harris  1974;  Saito  and  Harashima  1981;  Inouye  et  al.  1983;  Mars  and  Lopes  da  Silva 
1983;  see  Chapters  6.  10,  and  11  in  Voi.  1);  correlation  (see  Fig.  10;  reviewed  in 
Livanov  1977;  Gevins  et  al.  1981.  1985;  see  Chapter  6  in  Voi.  1);  and  multichannel 
non-stationary  autoregressive  modelling  (Gersch  et  al.  1983;  see  Chapter  10.  Voi. 
1). 

Onlv  a  few  of  these  methods  are  applicable  to  very  brief  data  segments,  which 
is  necessary  for  resolution  of  the  split-second  stages  of  neurocognitive  processing. 
Sometimes  it  is  possible  to  use  sets  of  short  data  segments  from  several  trials 


Fie  HI  Measures  of  the  similarity  in  (he  shape  of  event-related  potential  components  recorded  from 
different  channels  have  been  used  to  study  scatiotemporal  interdependencies  between  neural  systems. 
In  this  illustration,  the  most  significant  event-related  covanance  patterns  of  7  nem-handed  adults 
during  an  interval  from  50*3  to  S75  msec  after  the  presentation  of  a  cue  are  snown.  The  cue  prepared 
suoiects  to  mane  a  response  with  the  right  or  left  hand;  a  visual  stimulus  appearing  1  sec  after  the  cue 
indicated  tne  magnitude  of  tne  required  finger  pressure.  The  thickness  of  a  line  is  proportional  to  the 
necative  log  of  its  'igmficance  from  0  05  to  0  00?  Tie  color  af  a  iine  indicates  the  time  at  which  two 
channels  are  maximally  synchronized  t  yellow  tM5  msec;  green:  16-31  msec:  blue:  37—17  msec;  red: 
■Ui-7d  msec,  purple  >  hO  msec  I.  These  preparatory  patterns  are  focused  primarily  on  ieft  pretrontal 
and  midline  precentral  electrodes  for  performance  by  potn  hands,  and  central  and  parietal  electrodes 
acproortatelv  contralateral  to  the  hand  to  be  used  for  the  response.  Since  there  was  no  sign  of  overt 
muscle  or  eve  movement  activitv.  these  patterns  suggest  that  preparatory  sets  may  be  composed  of 
cognitive,  integrative-motor,  somesiheiicmotor  and  visuoperceptual  components.  For  details  aoout 
covjriance  patterns  associated  with  suosequemlv  accurate  or  inaccurate  pertormance.  see  Gevtns  et  al. 
i  Who  i 
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which  presumably  contain  consistent  signals  of  a  particular  process.  Furthermore, 
for  a  measure  of  ‘functional  coordination'  to  have  anatomical  and  physiological 
validity,  it  must  be  independent  of  reference  location  and  should  provide  infor¬ 
mation  about  the  temporal  relationship  of  inter-channel  signals,  that  is.  whether 
the  event-related  signals  of  one  channel  lead  or  'predict'  those  of  another  channel. 

11.6.4.  Localization  of  neural  sources 
11.6.4.1.  General  considerations 

Determining  the  sources  of  scalp-recorded  brain  activity  is  a  difficult  problem, 
especially  in  the  case  of  'endogenous'  signals  related  to  cognition.  The  use  of 
‘equivalent  current  dipole  model’  techniques  to  infer  the  neural  sou.ce  of  a  cani¬ 
cular  pattern  of  electric  or  magnetic  fields  observed  at  the  scalp  is  appropriate  only 
in  special  instances  which  conform  to  the  assumptions  of  the  model  (Kavanagh  et 
al.  1978;  Darcey  et  ai.  1980;  Ary  et  al.  1981;  Nunez  1981;  Williamson  and  Kaufman 
1981a. b;  Cohen  and  Cuffin  1983;  Okada  1983;  see  Chapter  9  in  this  volume,  and 
Chapters  13  and  14  in  Vol.  1  regarding  dipole  modeling  for  the  visual  system). 

The  assumption  of  single  generators  seems  particularly  unrealistic  in  the  case  of 
higher  cognitive  functions,  which  involve  the  integrated  activity  of  a  number  of 
neural  areas  (Luria  1970).  Since  the  areas  involved  can  be  widely  distributed  ana¬ 
tomically  and  vary  in  orientation,  it  is  not  usually  possible  to  derive  a  unique 
source  corresponding  to  the  complex  electrical  and  magnetic  field  distributions 
recorded  at  the  scalp.  Multi-source  solutions  may  be  possible  in  some  instances  by 
using  anatomic  constraints  derived  from  knowledge  obtained  from  studies  of  focal 
brain  lesions  and  radiological  metabolic  techniques. 

Another  problem  is  the  distortion  of  electrical  fields  as  they  pass  through  the 
tissues  between  cortex  and  scalp,  resulting  in  spatial  smearing  of  low  frequency 
potentials  and  attenuation  of  higher  frequency  activity  at  the  scalp.  The  decree  of 
distortion  varies  among  scalp  locations  and  persons.  This  distortion  can  be  correc¬ 
ted  by  2 -dimensional  spatial  deconvolution  (Nicholas  and  De  Loche  1976;  Hjorth 
198Z)  and  practical  techniques  for  implementing  this  are  being  developed  (Doyle 
and  Gevins  submitted). 

The  magnetic  fields  of  the  brain  are  relatively  undistorted  by  the  tissues  between 
cortex  and  scalp,  and  plausible  3-dimensional  equivalent  dipole  localizations  of 
certain  highly  localized  sensory  and  motor  sources  have  been  achieved  even  with 
the  relatively  crude  devices  currently  available  (reviews  in  Erne  et  al.  1981;  Wil¬ 
liamson  and  Kaufman  1981a. b;  Kaufman  and  Williamson  1982;  Okada  1983; 
Weinberg  et  al.  1985).  Magnetic  fields  measured  at  the  scalp  are  generated  prima¬ 
rily  by  tangentially  oriented  sources,  rather  than  the  mixture  of  radial  and  tangen¬ 
tial  sources  of  scalp-recorded  electrical  fields.  Thus  the  magnetoencephalogram 
provides  complementary  information  to  the  electroencephalogram,  and  a  great 
advance  in  source  localization  can  be  expected  when  technological  developments 
permit  the  simultaneous  recording  of  many  magnetic  and  electric  channels. 
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11.6.4.2.  Sources  of  cognitive  signals 

Exogenous  ERP  components  which  accompany  sensory  stimuli  and  motor  actions 
have  patterns  of  scalp  localization  corresponding  to  sources  in  primary  and  secon¬ 
dary  sensory  cortex  and  motor  regions  (Goff  et  al.  1978;  Wood  et  al.  1980,  1982; 
Allison  1984;  also  see  neuromagnetic  references  in  previous  section).  The  N100 
peak  is  maximal  at  occipital  sites  for  visual  stimuli  and  midline  central  sites  for 
auditory  stimuli  (presumably  projected  from  tangential  auditory  cortex  generators 
in  the  temporal  lobe).  Somatosensory  and  motor  ERPs  also  have  topographies  in 
agreement  with  neuroanatomic  and  neurophysiologic  localizations.  However,  no 
clear-cut  cortical  or  subcortical  sources  have  been  determined  for  endogenous 
components  such  as  the  N200,  P300  and  slow  wave.  In  the  case  of  P300.  intracranial 
recordings  from  neurosurgery  patients  suggest  that  it  arises  from  multiple  subcor¬ 
tical  sources  (discussed  in  Wood  et  al.  1980,  1982;  Squires  et  al.  1983).  In  one 
study  (Halgren  et  al.  1980),  a  strong  P300-iike  wave  was  observed  to  reverse, 
polarity  between  different  locations  in  the  hippocampus,  suggesting  an  active  gene¬ 
rator  there,  but  its  peak  occurred  after  the  P300  recorded  simultaneously  from  the 
scalp,  which  weakens  this  inference.  Polarity  reversals  of  P300-Iike  waves  have 
also  been  recorded  near  the  thalamus  (Yingling  and  Hosobuchi  1984),  and  between 
scalp  and  nasopharyngeal  electrodes  located  beneath  the  brain  (Perrault  and  Picton 
1984).  However,  the  absence  of  frontal  and  parietal  P300s  in  patients  with  anterior 
and  posterior  conical  lesions  argues  in  favor  of  cortical  involvement  (Knight  et  al. 
1980).  Magneioencephalographic  studies  suggest  sources  in  the  hippocampus 
(Okada  et  al.  1982),  but  the  evidence  is  somewhat  inconsistent,  and  it  is  possible 
that  this  technique  favors  sharpiy  localized  hippocampal  activity  over  diffuse,  dis¬ 
tributed  conical  activity. 

11.6.5.  Neurocognittve  models 

Most  neurocognitive  studies  are  based  on  very  simple  models  of  neurocognitive 
processing.  For  example,  in  EEG  studies  it  is  assumed  that  neurocognitive  proces¬ 
ses  are  relatively  stable  states  which  can  be  characterized  by  patterns  of  spectral 
intensities  (the  energy  in  various  frequency  bands  over  epochs  from  several  seconds 
to  minutes  long).  This  is  unrealistic  in  view  of  the  spiit-stcond  timing  of  the  stages 
of  cognitive  processing  indicated  by  behavioral  and  ERP  studies. 

A  more  dynamic  view  of  neurocognitive  processing  is  assumed  in  ERP  and 
ERMF  studies.  In  clinical  use  of  the  very  early  ’far-field’  evoked  potentials  to  test 
the  integrity  of  sensory  pathways,  there  does  appear  to  be  a  direct  correspondence 
between  successive  peaks  and  ascending  anatomical  structures,  but  the  situation 
with  higher  perceptual  and  cognitive  functions  is  more  complex.  In  cognitive  ERP 
studies,  it  is  usually  assumed  that  once  a  task  has  been  learned,  the  brain  processes 
stimuli  and  generates  responses  in  a  deterministic  manner.  The  successive  peaks 
and  waves  of  the  averaged  ERP.  or  the  principal  components  corresponding  to 
them,  are  assumed  to  reflect  the  stages  of  task  processing,  and  between-condition 
changes  in  amplitude  or  timing  of  a  peak  are  taken  as  indices  of  the  experimental 
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manipulation  of  a  particular  cognitive  function  and  specific  neural  sources.  The 
contribution  of  trial-to-trial  variability  to  such  amplitude  and  latency  changes  is 
not  usually  considered.  Although  cognitive  ERP  studies  continue  to  produce  inter¬ 
esting  results,  they  are  limited  by  the  unrealistic  simplicity  of  their  implicit  models 
of  neurocognitive  processing  and  the  resultant  assumption  that  these  complex  pro¬ 
cesses  can  be  indexed  by  a  few  amplitude  and  latency  measurements.  Similar 
effects  in  an  ERP  component  can  often  be  produced  by  different  experimental 
manipulations,  and  the  behavior  of  a  component  in  one  paradigm  frequently  con¬ 
flicts  with  its  behavior  in  another  when  interpreted  within  the  context  of  a  simplistic 
model. 

In  developing  better  models  for  future  research,  it  may  be  important  to  consider 
that  the  brain  systems  which  generate  event-related  signals  may  not  be  different 
from  those  which  produce  ongoing  EEG  activity.  It  is  possible  that  the  task  stimu¬ 
lus  induces  a  reorganization  of  ongoing  activity  in  a  neural  system  as  a  subset  of 
its  population  becomes  engaged  in  event-related  processing  (Sayers  et  al.  1974: 
Ba§ar  1980).  This  reorganization  may  be  evident  as  pre-  to  poststimulus  changes 
in  amplitude,  frequency,  timing  and  inter-electrode  coordination  of  low  frequency 
macropotentials.  An  appropriate  model  might  combine  the  'stimulus-evoked  activ¬ 
ity'  and  'pre-  to  poststimulus  reorganization  of  activity"  approaches,  and  would 
have  the  advantage  of  taking  into  account  the  prestimulus  activity  related  to  expec¬ 
tancy,  preparation  and  attentional  set.  Further,  because  of  the  brain's  complex 
hierarchical/parallel  organization  and  flexibility,  the  same  overt  behavior  in  a  cog¬ 
nitive  task  might  be  associated  with  different  configurations  of  regional  activity, 
especially  through  the  learning,  skilled  performance  and  'automatized’  phases  of 
an  experiment  (John  et  ai.  1977a:  Thatcher  and  John  1977).  Improved  analytic 
models  should  allow  for  such  possible  variation  in  functional  organization. 

Recent  studies  in  animals  are  paving  the  way  for  improved  models  of  neural 
processing  in  humans.  Freeman  and  colleagues  have  demonstrated  that  spatial 
patterns  of  macropotentials  in  the  olfactory  system  of  rabbits  contain  information 
not  present  in  the  Firing  of  single  neurons,  that  these  patterns  are  'neural  templates' 
evoked  during  conditioned  expectation  and  perception  of  specific  odors,  and  that 
there  are  correlations  between  the  macropotential  patterns  of  olfactory  bulb  and 
cortex  (Freeman  1975.  1979a. b.c.  1981,  1983;  Bressler  1984,  1986;  see  Chapter 
18  in  Vol.  1).  It  is  a  large  leap  from  rabbit  paleocortex  to  human  neocortex,  but 
similar  principles  of  mass  neural  organization  may  also  apply  at  the  more  macro¬ 
scopic  level. 

Understanding  of  human  neurocognitive  processes  will  advance  in  pace  with 
improvements  in  recording  and  analytic  technology.  The  next  generation  of  re¬ 
search  will  probably  view  neurocognitive  processing  as  a  rapidly  shifting,  distrib¬ 
uted  network  of  integrated  activity  in  many  conical  and  subconical  areas.  The 
First  step  toward  development  of  such  a  model  will  be  the  measurement  of  some 
sign  of  functional  coordination  between  neural  areas  during  cognitive  tasks.  In  the 
coming  decade  we  might  expect  to  achieve  a  high  degree  of  spatial  resolution 
through  recordings  of  more  than  200  electric  and  magnetic  scalp  sensors  combined 
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with  advanced  analytic  techniques  such  as  spatial  deconvolution.  In  some  cases, 
this  may  include  3-dimensional  localization  of  multiple,  simultaneously  active 
sources. 
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Appendix 

The  probe  ERP  method  is  an  attempt  to  obtain  information  about  regional  brain 
activity  associated  with  cognitive  processes  from  ERP  waveforms  elicited  by  irrele¬ 
vant  ‘probe'  stimuli  presented  at  various  times  during  a  task.  This  is  an  adaptation 
of  the  probe  reaction-time  method  used  in  psychological  studies  to  assess  the 
commitment  of  limited-capacity  processing  to  a  primary  task.  The  basic  assumption 
of  the  probe  ERP  method,  the  ‘ceiling-  or  ‘limited  capacity'  hypothesis,  proposes 
that  the  neural  response  to  a  probe  stimulus  is  reduced  by  the  commitment  of  a 
local  neural  population  to  primary  task  processes  (reviewed  in  Papanicolaou  and 
Johnstone  1984).  For  example,  the  right-sided  lateralization  of  P100  and  N 100 
peaks  for  visual  probes  during  an  auditory  linguistic  task  (relative  to  a  non-verbal 
task)  was  due  to  lower  amplitudes  over  left  hemisphere  ('language')  sites  (Papani¬ 
colaou  1980).  Reduced  probe  ERP  amplitudes  over  left  hemisphere  sites  have  also 
been  measured  during  mental  arithmetic  (Rasmussen  et  al.  1977)  and  visuospatial 
tasks  (Papanicolaou  et  al.  1983). 

The  significance  of  the  brain's  response  to  non-contingent  probes  and  the  effects 
of  probes  on  task-related  processes  have  not  been  determined.  If  the  probe  ERP 
is  affected  by  task  processes  (via  the  ‘ceiling  effect'),  then  probe  stimuli  may  in 
turn  affect  those  task  processes.  Questions  about  the  effect  of  probes  on  conical 
activation  and  the  habituation  of  responses  to  frequently  presented  probes  are 
raised  by  observations  of  regional  changes  in  ERP  amplitudes  and  EEG  spectral 
coherence  associated  with  levels  of  autonomic  arousal  and  stages  of  conditioning 
(Gershuni  et  al.  1960:  Khachaturian  and  Gluck  1969;  Khachaturian  et  al.  1973: 
Hudspeth  and  Jones  1975).  Habituation  of  probe  ERP  amplitudes  was  evident  in 
a  study  companng  auditory  linguistic  and  non-linguistic  tasks  (Shucard  et  al.  1977). 
Probes  consisting  of  pairs  of  tone  pips  2  sec  apart  elicited  task-related  asymmetries 
due  to  larger  ERP  amplitudes  for  the  second  tone  over  the  hemisphere  presumed 
to  be  involved  in  each  task.  This  was  attributed  to  a  difference  in  the  ‘habituation 
recovery  cycle'  between  engaged  and  non-engaged  hemispheres,  implying  that  the 
engaged  hemisphere  was  'too  busy'  to  habituate.  It  is  not  clear  how  this  result 
relates  to  the  'ceiling  hypothesis'  which  proposes  that  probe  amplitudes  will  be 
reduced  over  ‘engaged'  areas. 

Another  consideration  is  the  effect  of  the  physical  properties  of  probe  stimuli. 
Lateral  ERP  asymmetries  have  been  observed  even  for  simple  unstructured  stimuli 
(Friedman  et  al.  1978).  and  opposite  asymmetries  to  auditory  and  visual  probes  in 
the  same  task  have  been  reported  (Johnstone  1982).  Also,  the  ERP  elicited  by 
task-irrelevant  probes  consists  mainly  of  exogenous  peaks,  which  are  thought  to 
be  generated  by  primary  sensory  processes,  and  the  nature  of  the  interaction  of 
such  sensory  processing  with  higher  cognitive  processes,  which  may  involve  diffe¬ 
rent  neural  populations,  is  not  known. 

It  has  been  proposed  that  the  probe  ERP  method  has  the  attractive  advantages 
of  ‘ecological  validity'  and  freedom  from  the  need  to  control  stimulus-  and  respon¬ 
se-related  factors  (Papanicolaou  and  Johnstone  1984).  This  would  mean  that  the 
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probe  method  might  be  applied  in  more  naturalistic  Teal-life’  situations,  without 
the  artificial  constraints  imposed  by  well-controlled  ERP  experiments.  However, 
upon  reflection,  it  is  dear  that  the  probe  method  does  not  avoid  the  need  to 
control  stimulus-  and  response-related  factors  and  artifacts,  since  these  factors 
have  strong  effects  on  brain  potentials  which  can  easily  be  confused  with  the  effects 
of  cognitive  factors  per  se.  If  probe  ERPs  are  sensitive  to  local  neural  activity,  they 
will  certainly  be  affected  by  neural  responses  to  such  stimulus  and  response  factors 
and  will  also  be  affected  by  eye  and  muscle  artifacts,  since  random  timing  of  probes 
cannot  eliminate  the  effects  of  such  activity  which  would  be  present  in  naturalistic 
situations. 

Randomly  limed  probes  do  not  provide  precise  information  about  the  split-se¬ 
cond  stages  of  cognitive  processing.  The  ‘background  information  probe'  method 
(Thatcher  and  Maisel  1979)  is  a  more  controlled  approach  which  uses  probe  stimuli 
consisting  of  random-dot  patterns  flashed  at  precise  times  during  a  task.  In  a  study 
of  delayed  matching  of  word  pairs,  the  ERP  waveforms  to  probes  presented  at 
different  times  were  distinguishable  by  factor  analysis.  However,  the  word  stimuli 
themselves  provided  more  information  about  the  localization  of  linguistic  proces¬ 
ses.  since  the  most  pronounced  asymmetries  occurred  over  the  posterior  ‘language' 
(Wernicke's)  area  in  response  to  the  second  word.  It  is  possible  that  with  appro¬ 
priate  refinements,  the  probe  ERP  method  could  provide  useful  information,  but 
many  issues  must  be  resolved  before  it  can  be  used  with  confidence. 
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